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Abstract 



With ilM implementation of the FaU Term 1990 pUot of the Student Feedback Survey, 
M-DCC joined the majority of institutions of higher education in formally requesting 
information from students on their classroom experiences. The process invoNed randomly 
selecting one section for each full-time faculty member and administering the surveys prior 
to the final course withdrawal date. Surveys were returned from 640 sections (or 84% of 
those initiaUy selected) and included responses from 12,729 students. 

Results showed that students were generally pleased with their classroom experiences. 
At least 95% of students thought their instructor was prepared for class, showed an interest 
in and knowledge of the subject, and treated them with respect The same percentage 
agreed their instructor distributed the course objectives and discussed the grading system 
with them. The only item where students gave significantly lower ratings was in providing 
regular information about their progress; only 62% of students agreed or strongly agreed 
with this item. 

Students described themsebes as generally hard-working, dedicated to class 
attendance and performing well in class. Most (75%) were working at least part-time and 
30% indicated they had family commitments that interfered with school. Most (80%) 
thought the class had about the right number of students. A majority (71%) were in the 
course because it was required. They generally approved of the survey process. 

Other studies have found that students see good teaching as involving a variety of 
factors or components-that the instructor is not just "good" or "bad". Results showed that 
M-DCC students made similar distinctions. Using factor analysis, it was found that students 
tended to group items along eight dimensions (or factors). The factors were labelled (1) 
Focus on the Individual, (2) Competence in the Classroom, (3) Approach to the Material, 
(4) Gra'ding Policy, (5), Listening to Students, (6) Qarity of Course Objectives, (7) Fairness 
of Examinations, and (8) Active Learning. 
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Instnictors who were generally rated highly overall were most likely to be described 
as creating a classroom atmosphere that encouraged learning (item 14), being concerned 
with students* progress (item 5), and making the course interesting (item 7). In addition, 
these instructors taught what ibty said they were going to (item 4), graded their examina- 
tions fairly (item 11), and paid attention to student comments (item 20). 

Much of the stucfy focused on reliability and validity issues. Results showed the 
reliability for the survey was very high - .94 for the 23 items used to rate instnictors and 
courses. Factor reliabilities ranged from .66 to .84. Validity questions focused on testing for 
rating changes based on student perceptions of their classroom performance, course 
difficulty, course workload, class size, reasons for taking the course, time of day the course 
was taught, instructor rank, and subject The largest differences were found for student 
performance, instructor rank, and subject area. 

On the issue of student classroom performance, results showed that students who 
rated their performance as "good" or "excellent" gave higher ratings on all eight factors than 
students who did not feel they were doing as well in class. Perhaps those who were doing 
well in class found the instructor more effective than those who felt tl»y were not 
understanding the material Perhaps the expectation of a high grade produced higher 
evaluations. In general, however, other research has indicated that amount of learning 
rather than grades was a better indicator of instructor ratings. 

Differences in ratings were also found based on professional rank for four of the 
factors: (1) Focus on the Individual, (3) Approach to the Material, (5) Listening to 
Students, and (8) Active Learning. Instructors and assistant professors received higher 
ratings than other ranks. Again, several interpretations are possible. One is that those who 
are "freshest" to M-DCC carry that freshness into the classroom. Miller (1987), for example, 
found that faculty received their highest ratings between their 3rd and 12th years of teaching. 
Another possibility is that another variable related to faculty rank is actually responsible for 
the difference. For example, some departments or subject areas which students enjoy more 

S 



may have a greater proportion of new faculty and it is the subject area causing the 
differences rather than faculty rank per se. 



Indeed, differences based on subject matter did emerge from the analyses on three 
factors. Students enroUed in EngUsh-as-a-Second-Unguage (ENS), English (ENC), and 
Nursing (NUR) courses provided higher ratings in the area of Focus on the Individual than 
Mathematics (MAT and MAC). AppUed Biology (APB). Humanities (HUM), Chemisti> 
(CHM), and Psychology (PSY). In the area of Active Learning, students rated Chemistry, 
English, Mathematics, Nursing, and English-as-a-Sccond Language c»urses higher than 
Applied Biology, Humanities, and Psychology. Fmalfy, students perceived Matiiematics 
examinations as fairer tiian English examinations. While some of the differences must surely 
be due to the content (e.g., hard science vs. the humanities), others are not as readily 
explained. 

Statistical significance was found for some other variables, but the mean differences 
were not as great as for those already mentioned. These include course difBcully, amount 
of work required, class size, and reasons for takuig the course. There were no differences 
when ratings were viewed based on the time of day the course was offered. 

In summary, the survey appears to meet acceptable standards for reliability and 
validitv. Later research will focus on confirming the factor structure and further explaining 
rating differences. Readers are urged to study the full report for details and campus-level 
information. 
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Reliability and Validity Issues: 
An Analysis of Miami-Dade*s Pilot Student Feedback Survey 

Rackpround and Introduction 

Despite all the furor, student evaluations of instruction arc a part of academic life at 
most institutions of higher education in the United States. In the Fall of 1990, Ndiiami-Dade 
Community College joined the majority by piloting a survey and a process for gathering 
feedback from students. 

The Student Feedback Survey came about as a result of the Teaching/Learning 
Project The Teaching/Learning Project began in 1986-87 with the ^)als of improving the 
quality of teaching and learning at M-DCC making teaching a professionally rewarding 
career, and making teaching and learning the focal point of college activities and decision- 
making processes. As an initial step in the process, the Faculty Excellence committee 
developed a series of statements of excellence in teaching rooted in empirical studies on 
student learning. These statements then became the basis for the survey development work 
of the Faculty Advancement committee. Thus, the Student Feedback Survey became one 
piece of a larger whole in focusing on teaching and learning at the College. 

The body of research on student ratings of instruction is voluminous. Perhaps this 
is because student evaluations are seen as having more influence on tenure and promotion 
decisions than any other data source (Miller, 1987). There is justifiable concern, therefore, 
that student ratings accurately reflect the quality of instruction provided. Thus, most 
research has focused on whether students can make distinctions in evaluating instruction, 
whether their ratings are re'iable, and which factors, if any, can bias student ratings. 

Student ratings of instruction at Miami-Dade will be included in annual performance 
reviews"anc{« become part of performance portfolios that faculty will present for promotion 
and tenure decisions. Here, too, then it becomes important that student ratings provide an 
accurate reflection of faculty performance. 
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The purpose of this study is to answer some basic questions about the survey results. 
The focus is mainly on reUabiUty and validity issues. SpcciGcally, the questions for this study 
are: 

1. How do students generally feel about the courses and faculty they arc 
encountering? Satisfied or dissatisfied? On which items are students providing 
the highest and lowest ratings? 

2. How do students generally describe themselves? What are the general 
characteristics of students who responded to the survey? 

3. How do the items relate to one another? If students give a high rating on one 
item, win they give the same high rating on all the other items or do they 
respond to each item separately? Do student have one general construct that 
they arc using in their ratings or are there a series of underlying constructs 
they are using? 

4. How reliable is the survey? Do the items form a stable and coherent whole? 

5. Do student ratings change based on: 

a. perceived performance in the class? 

b. the difficulty of the course? 

c the amount of woik required by the course? 

d. the number of students in the class? 

e. reasons for enrollment? 

f. time of day the course is offered? 

g. the rank of the faculty member? 

h. the subject matter? 

6. What do students think of the survey and the process itself? Is the survey 
understandable? Too long or too short? What comments did they make 
about the survey? 



Methodology 



For the Fall, 1990, term the process implemented by the Faculty Advancement 
Procedures 5 subcommittee involved randomly selecting one course for each full-time 
classroom faculty member and distributing answer sheets and surveys to the faculty member 
for completion in that course. Surveys were administered prior to the final date for course 
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withdrawal (November 14-26, 1990). Of the 760 ftill-tinM faculty who were teaching in the 
faU term, surveys were processed for 640 or 84%. 

The survey for the Fall Term consisted of 43 items, along with spaces for comments 
on the survey and comments directed to the feculty member. The first 23 items were related 
to the instructional process. For these items, students had four choices in responding: 
strongly agree (1). agree (2), disagree (3), or stronrtr disagree (4). The remaining 20 items 
requested smdcnt information related to classroom performance, perceptions of course, eta 
Many of the items in the student demographics section were used to look for changes in the 
23 items on the instructional process. A copy of the survey can be found in Appendix A. 

This study used two files that contained survey results. One contained the responses 
for each student for each course; a total of 12,729 surveys were included in this Fall Term 
file. The second file contained course level information and was based on the 640 sections 
which had survey data processed. Note that because classes were randomly selected, 
students might fill out surveys in more than one course. Full-time students, of course, would 
be more likely to complete multiple surveys than part-time students, because they take more 
courses and because full-time faculty are more likely to be teaching at times when full-time 
students are enrolled (i.e., during the day). 

A combination of statistical procedures was employed to analyze the data. Readers 
interested in these details are urged to turn to Appendix B. Summaries of item responses 
were based on everyone who responded to each item. Most statistical analyses, however, 
eliminated anyone who failed to answer one or more of the items. 

Analysis of Results 

What Do Stud ents Think of Their Courses and Faculty? 

Students were generally pleased with their classroom experiences. In particular, over 
95% thought that their instructor was prepared for class and showed an interest in and 
knowledge of the subject In terms of classroom procedures, 95% agreed the Instructor 
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distributed tlw course objectives and discussed the gradtng system at the beginning of the 
semester. Finaify, on the interpersonal level, 95% thought tte instructor treated them with 
respect 

There was only one item where a significant number of students foiled to agree. Only 
62% of the students agreed that the instructor informed them regular^ about their progress. 
Some faculty have indicated that they were unclear about what students meant when rating 
this item since even in sections where they gave week^ quizzes and returned them the 
following class period, students rated th» item lower than others. 

Other items on which more than 15% of the students disagreed included: Tiie 
instructor is concerned with my progress (18% disagreement), the ir^tructor shows me how 
the course material can benefit me beyond the classroom (17%), the instructor makes this 
course interesting (17%), and the instructor uses a variety of teaching methods (17%). 

FuU results, including the number and percentage of students selecting each item 
response, are included in Table 1. Results by campus can also be found in this table. 

Mean results are shown in Table 2. Note that the means were calculated where 
strongly agree- 1, agree=2, disagrce«3, and suongly disagree«4. The lower the numbers, 
therefore, the more positive the responses. Findmgs paralleled those for Table 1. 

Hnw Do StudpnK Describe ThemseWes? 

Students tend to see themselves as a fairly dedicated and hard-working ^oup. As 
shown by Table 3, 85% report they almost always come to class, 63% are abnost always 
prepared for class, and 72% almost always pay attention. About two-thirds (67%) say they 
are almost never late. Faculty may find these perceptions somewhat more optimistic than 
their own. 

♦ 

A majority of students thought they were doing well in the class being surx'eyed. 
About 18% rated their performance as "excellent" and 51% rated it "good". If a rating of 
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-excellent- corresponds to an "A" and a rating of -good" corresponds to a "B" or "C. then 
student perceptions of their performance in their classes is somewhat higher than the grade 
distribution at the end of the term would indicate. According to the most recent analysis of 
grade distributions (see Vorp, 1990) for all courses, 19% were awarded "AV and 41% were 
awarded "BV or "CV. 

Most respondents thought the course diCBculty and workload were similar to other 
courses they had taken. About 50% rated course difficulty about the same while 56% rated 
the amount of work about the same compared to other courses. If students die* not think 
the course was similar in these respects, they were more likely to rate the course as more 
difficult (29%) and requiring more work (28%) than they were to see the course as easy. 
Most (71%) were enrolled in the course because they were required to do so. A few (14%) 
had previously registered for the course. 

Most students were pleased with ttic size of their classes. A total of 80% thought 
there were the "right number" of students in their class, while 17% thought there were too 
many. Only 3% indicated there were "too few" students in the class. We can only speculate 
who might select this option, but ii is possible that students who preferred not to actively 
participate in class would select this option. 

Respondents were also asked to identify themselves by gender, age, and racial and 
ethnic heritage. Compared to the ' ill, 1990, profile of all students (see Morris, 1991), it 
appears that males and females were proportionally represented and younger students were 
somewhat over-represented in the survey. While approximately the same proportions 
identified themse^cs as "Hispanic" for the Fall Profile and on the survey, a disproportionate 
number of students (21%) identified themselves as "other" on the survey when racial groups 
were listed. Thus, ethnic identifications arc somewhat suspect on the survey. Recall too, 
thai students may have completed more than one survey. 

I 

The survey included a series of questions designed to discover how much time and 
effort students were giving to courses and school Slightly over 30% said they had family 



commitments that intcifcred with how weU they did in class. About 75% were working at 
least part-time, and 10% were working more than 40 hours per week. Yet a majority (63%) 
indicated they were ftiU-time students, enrolling for 12 or more credits. This figure is 
different from the data in FaU Profile, which indicates that only 36% of students were 
enrolled full-time, and confirms that full-time students had more opportunities to complete 
multiple surveys. 

Further details on student characteristics can be found in Table 3. 

What Dimensicnf f^tV^mff R^^"?? 

Most of us would agree that the act of teaching is a complex activity involving many 
dimensions, A faculty member may be veiy knowled^able about the subject matter but be 
unable to project enthusiasm to the students. Another may be enthusiastic but disorganized. 
The question to be addressed in this section is whether students, too, view the instructor and 
classroom across a variety of dimensions. If they do, what are those dimensions? Or do 
students decide their classroom experience was either good or bad and engage in halo 
ratings with all items uniformly high or low regardless of content? 

In the large body of research on student evaluations of instruction, there is general 

agreement that students can and do make distinctions in their ratings. Marsh (1991) found 

nine factors or dimensions in his 35.item survey. He named them Learning/Value, Instructor 

Enthusiasm, Organization/Qarity, Group Interaction, Individual Rapport, Breadth of 

Coverage,- Examinations/Grading, Assignments/Readings, and Workload/Difficulty. In a 

review of 11 studies, Kulik and McKeachie (1975) identified four common dimensions: Skill, 

Rapport, Structure, and Overload Cohen (1981) used six dimensions in his work: the four 

from Kulik and McKeachie plus two others he identified as Interaction and Feedback. 

Much of this research, however, has been conducted at four-year institutions. Do community 

college students make the same kinds of distinctions? 
« 

Factor analysis was used to answer this question. Basically, what this procedure does 
is to look at the correlations among the items and determine which items (if any) group 
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together separately from other items and form what can be referred to as a "factor" or 
"dimension" or "construct". Readers interested in more technical details on this process are 
referred to Appendix B . 

Results showed that M-DCC students did make distinctions. In fact, based on the 
Fall results, it appears that students had eight different underlying constructs in mind as they 
completed their rating forms (see Table 4). Based on the items included in each of the 
factor groupings, the factors were tentatively named as follows: 

1. Focus on the individual 

2. Competence in the classroom 

3. Approach to the material 

4. Grading policy 

5. Listening to students 

6. Qarity on course objectives 

7. Fairness of examinations 
8w Active learning 

The items which were most strongly related to that factor are included under the 
name of the factor and were used to help name the factor (see Table 4). The number or 
weighting next to the item under each factor shows how much weight the item has in that 
factor. The weighting can be thought of as a correlation between that item and the factor 
and can range from an absolute valve of 0.0 to 1.0. Though we could include the weights 
of every item for every factor, we only included those that were .30 or higher to facilitate 
interpretation. 

Factor 1 (Focus on the IndividuaO included four items such as 'The instructor is 
concerned with my progress" and 'The instructor informs me regularly about my progress." 
Factor 2 was titled " Cnmpetence in the Qassroom " because it included items on instructor 
interest in the subject, preparation, and subject knowledge. Factor 3 (Approach to the 
Material '^ included items such as 'The instructor creates a classroom atmosphere that 
encourages *jmc to iearn" and 'The instructor makes this course interesting". Factor 4 
(Grading Policy ) was based on two iteins: 'The instructor discussed the grading system at 
the beginning of the semester" and 'The instructor mad? the grading system clear to me." 
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EaclfiLi had some similarities to Factor 1 in the types of itenas included. This factor 
was named ^.u^^nin y to Students" and included items such as 'The instructor pays attention 
to my comments" and The instructor treats me with respect" Another name for this factor 
might be "Studem Rapport". Ymnx QmPf Qf rmm O^ffatVgS^ ^ based on two 
items: The instructor distributed the course objectwcs/competencies" and 'There is 
agreemem between the objectives/competencies of this course and what is taught" fafilQT 
2. entitled "EaimC5iil£ffiminatos". included the items The examinations and/or other 
forms of evaluation are related to the course roatcrwl- and The examinations and/or other 
forms of evaluation are graded fairly." A third item on the grading system appeared under 
this factor as well as under Factor 4. The last factor, EastOtS. was called "AsXistLfiSminft" 
because it included the items "Assignments help me learn the course material" and 'The 
instructor encourages me to think for myself." 

Although the factors are presented as separate dimensions, they are correlated with 
one another the same way that the items are. Table 5 shows that the correlations among 
the factors arc fairly strong, ranging from .44 to .66. The most important factors (based on 
the variability attributed to each factor) are Factors 1 (Focus on Indhddual) and Factor 3 
(Approach to the Material). Thus, it can be said that how students respond to the items 
making up these two factors will influence how they respond to the items comprising the 
other factors. 

Uke each factor, the responses to each item contain variability that is unique to that 
item based on its specific wording, etc, and variabUity the item shares with other items. 
Table 6 displays the correlations of the items with each other and shows the extent of the 
common variance between each pair of items. Note that the correlations range from a high 
of .71 (between items 7 and 14) and a low of .22 (between items 6 and 23). The 
communalities at the bottom of the table show how much variability each item shares with 
all other items. One could think of items with high communalities as "linchpin" or core items 
that hold thi survey together. The top six items in this survey are: 

Item 14: The instructor creates a classroom atmosphere that encourages me to 
learn. (c=.73) 
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Item 5: The instructor is concerned with my progress. (ce.71) 

Item 7: The instructor makes this course interesting. (c=.70) 

Item 4: There is agreement between the objectives/competencies of this course 
and what is taught (c=.69) 

Item 11: The examinations and/or other forms of evaluation arc graded fairly. 
(c=.67) 

Item 20: The instructor pays attention to my comments. (c=.67) 

The item that was least related to the others was item 23, The instructor starts class 
on time." This also was the only item which was not included in any of the fectors. This 
item was changed for the Winter survey to "The instructor uses class time productively." 

How Reliable is the Survey? 

In any measurement process, the reliability or stability of the scores is the first 
question to be raised. If there is no assurance that the scores or ratings will stay fairly 
constant-if students change their ratings from occasion to occasion when there is no change 
in what is being measured-then there is no need to proceed any further. The ratings will 
have no meaning. 

There are a number of ways to measure reliability. One of the simplest is known as 
"internal consistency reliability." It measures the extent that the items are measuring the 
same thing. Results showed that the reliability of the survey was very high~.94 for the first 
23 items using Coefficient Alpha. The factors also showed satisfactory reliabilities in most 
cases: .79 for Factor 1, .66 for Factor 2, .84 for Factor 3, .73 for Factor 4, .77 for Factor 5, 
.73 for Factor 6, .75 for Factor 7, and .69 for Factor 8. Given the small number of items 
included in the factors, these reliabilities are fairly high, though the reliabilities for factors 
2 and 8 should probably be a little higher. Adding more items that are similar usually 
improves reliability. 



Va||rttiy Issues; What Makes Ratings ChanK? 

Once reliability has been established, researchers turn to questions of validity. 
Validity issues revoke around what interpretations can be placed on the ratings or scores. 
In this study, do fectore other than effectwe teaching influence the ratings that students give 
instructors? 

The number of questions that can be addressed in this area are almost infinite. For 
a first round, however, some basic characteristics of students, courses, and instructors were 
used to compare mean or average performance. Most of the qwstions included in this 
section were posed by the Faculty Advancement committee. Ethnicity and gender were 
excluded as variables because they were being addressed in another study (Ciereszko, 1991). 

To facilitate comparisons across factors with different numbers of items, the factor 
scores were standardized so each had a mean of zero (0.0) and a standard deviation of one 
(1.0). For this analysis, adjustments were made so that means above zero indicated more 
positive ratings compared to the norm, while means below zero indicated less positive 
ratings. 

Because of the large number of observations included, statistical significance could 
be found for small mean differences. Therefore, a rule of thumb was implemented for 
"educational significance." Besides being statistically significant, means had to vary by at 
least one-half standard deviation (0.5) to be considered educationally significant Since 98% 
of the individual ratings for each factor fell between -2.0 and +2.0 (or two standard 
deviations on either side of the mean), a mean difference of 0.5 was quite large. 

As a final check on educational significance, for those comparisons that were 
statistically significant, the correlation between group membership and the factor rating was 
calculated. When squared (R^), this indicates the amount of variability in the scores that can 
be explainedlby group membership. For further information on the methodology employed 
in this set of analyses, please refer to Appendix B. 

If) 



-10- 



« 



| >y Ratings Q iange Based ftfl ^rcBwed Qassroom Perfomiancc? 

The question of grades and tearning is imxtricably tied to ratings of teaching 
effectiveness. Some argue that all faculty need to do to improve their student evaluations 
is to grade more leniently (and perhaps assign less work). Others argue, however, that to 
the extent that grades reflect student learning and that students team more with more 
effective teachers, we should expect and even welcome a relationship between student 
grades or performance and ratings. 

At Miami-Dade, as part of the Fall pitot survey, students were asked to rate their 
performance in class as "excellent", "good", "fair", "poor", or "don't know". Results showed 
both statistically and educationally significant differences for all foctors (see Table 7). In 
every case, students who rated their performance as "excellent" or "good" gave higher ratings 
on all factors than students who rated their performance as "poor" or who didn't know how 
they were performing. Students who rated their performance as "fair" were similar to the 
higher performers on some factors (e.g. grading policy) and like the poorer performers on 
other factors (e.g., focus on the individual, approach to the material, and active teaming). 

The correlation between performance and rating also differed depending on the 
factor. The highest correlation (.31) was obtained for factor 1, Focus on the Individual, 
while the lowest (.16) was found for Factor 2, Competence in the Qassroom. 

These results are in line with findings from other studies. Most researchers have 
found ratings to be moderately correlated with grades (MOter, 1987). Marsh (1982, 1980), 
for example, found students gave more favorable rating when they expected higher grades. 
In a study of conmiunity college courses, however, Beatty and Zahn (1990) concluded that 
evaluations were not a product of easy grading practices. It should be noted that stronger 
relationships exist between ratings and achievement Cohen (1981), in a review of the 
research, concluded that students rated teachers highest that they learned the most from. 
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R nti nr rhangff ft H ^f^ Perceived Course Difficttltv? 
Students at M-DCXT were also asked if they thought the course they were rating was 
more difficult, less difficult, or about the same difficulty as their other courses. Again, 
statistical significance was found for each factor (see Table 8). With one exception, students 
who found the course more difficult gave lower ratings than the other two groups (who did 
not differ from one another). For Factor 3 (Approach to the Material), every group differed 
from every other group. In terms of educational significance, however, no differences 
appeared because the mean differences were too small Correlations were also smaUer, 
varying from .04 to .12. 

Ukc grades, it U hard to pull apart the meaning of course "difficully". How much of 
difficulty is due to the subject matter, the approach the teacher takes in presenting it, and 
student "preparedness" to grasp the material? Though Cohen (1981) general^ found no 
consistent and significant relationship between course difficulty and student achievement, 
others have found modest relationships between student ratings and course difficulty. Marsh 
(1982), for example, found that for the same instructor, the course that received the higher 
evaluation required more work. He found similar results in an earlier study (March, 1980). 
Brady (1989) asked community college students to read a series of scenarios and to rate 
hypothetical instructors. He found students in theory preferred a demanding, high grading 
professor over an easy, high grading one and saw the former as providing "higher quaUty". 

These results contradict those for Miami-Dade. It should be pointed out, however, 
that the outside studies combined "course difficulty" and Workload". Among other possible 
interpretations, "course difficulty" can mean that the student had to work very hard to 
understand the material but was able to with work, and/or the student received a lower 
grade in this course because of lack of comprehension of the material, and/or the course had 
a heavier workload than most but the student performed well. 

P rf <^hange Rased on Amount of Work Required? 

As previously noted, a course may not be difficult but may require a great deal of 
work from students. As part of the survey, students were asked to judge whether the course 

21 

-12- 



they were compteting the survey in was more work, less work, or about the same as their 
other courses. Again, statistical significance, but not educational significance, was obtained 
on all eight fectors. Gjrrclations were low, ranging from .05 to .11 

In most cases, students who thought the course required about the same amount of 
work as other courses gave the highest ratings. Students who thought the course was Isss 
work than their other courses were likely to give the lowest ratings. This group gave the 
lowest ratings on Focus on the Individual (Factor 1), Competence in the Classroom (Factor 
2), Grading Policy (Factor 4), Fairness of Examinations (Factor 7), and Active Learning 
(Factor 8). 

These findings are somewhat contradictory to the M-DCC results on course work- 
load/difficulty. What is similar, however, is the finding that requiring less in the class will not 
lead to higher ratings from students. 

Dn Ratings Change f^m^ on Qass Size? 

L43gic would dictate that smaller classes would receive more favorable evaluations. 
Research findings, however, have been mixed (Miller, 1987). Feldman (1984) reported that 
when relationships were found, size was typically related to evaluations in one of two 
patterns. The first pattern was a weak inverse relationship with higher evaluations 
associated with smaller classes. Bausell & Bausell (1979) found, for example, that when 
evaluations for two classes for the same instructor were considered, the larger class received 
lower evaluations. The second was a U-shaped pattern with higher evaluations associated 
with very large and very smaU classes; it was hypothesized that better teachers may have 
been selected to teach larger sections as a way of explaining this second pattern. 

In this study, class size was defined in two ways. One was to ask students whether 
they thought there were too many, too few, or about the right number of students in the 
class. The qther approach was to use the end-of-term course enrollments. While it probably 
would have been more accurate to look at course enrollment at the time of survey 
completion, unfortunately this information was not captured by the computer and retained 



for analysis. A correlation between class size at the end of the term and number of students 
completing the survey, however, indicated a strong relationship between these two measures 
(r=».90). 

Based on students' perceptions of class size, statistical significance, but not 
educational significance, was found for all eight factors. Generally, students who thought 
they had about the right number of students in their classes gave the most positive ratings. 
Students who thought there were "too few" students in their class gave the lowest ratings. 

A definition of what was "too many" or "too few" seemed to be extreme^r individualis- 
tic In only 20% of the classes surveyed did all students agree that the class size was about 
right An additional 21% of classes contained students who responded in all three 
categories~too many, right number, and too few. For students who felt there were too many 
enrolled, the median end-of-term class size was 28. The median dropped by 5 students to 
23 for those who thought the class contained about the right number of students. For those 
who thought too few students were enrolled the median class size was 21. 

Using end-of-term enrollment resulted in statistical significance on fewer factors than 
was found using student perceptions. Statistical significance was obtained for Factor 1 
(Focus on the Individual), Factor 5 (Ustening to Students), and Factor 8 (Active Uaming). 
Generally, large classes (over 30 students) were rated significantly lower than small classes 
(20 or less students). Qasses in the 20-30 student range fell in the middle. Because the 
means did not vaiy by .5 or more, none of these differences reached educational significance 
(see Table 11). Other researchers have also noted that class size seems related more to 
some factors than others, e.g. Group Interaction and Individual Rapport (Marsh, Overall, 
& Kesler, 1979; Frey, 1978). 

P9 R f^ ltngs Change B^^d nn Reasons for Taking the Course? 

Students responding to the survey were asked why they had enrolled in the course. 
Over 70% were there because it was required for their degree. Other reasons were divided 
among the remaining four options. 
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Again, statistical but not educational significance was obtaiiwd for each factor. 
Generally^ the most positive group of raters was the group who enrolled based on personal 
interest in the course. The most ne^tive group tended to be those who enrolled for "other" 
reasons. Those who enrolled because the course was required or an elective fell between 
these two extremes. See Table 12 for further details. These findings confirm Marsh's 
previous research (1980» 1982). 

nn Ratinps Chan^ Based on the Time of Dav the Course is Offered? 

There is a general perception that different types of students enroll in courses offered 
at different times of the day. Those who are older and employed are thought to take 
courses very early in the day, over lunch, and in the evenings. Full-time students seem to 
make up the bulk of the morning and afternoon enrollments. In addition, class sizes change 
with early morning classes tending to be the smallest At least one study confirms that 
afternoon classes receive lower ratings than morning classes (Nichols & Soper, 1972 as 
reviewed in Arubayi, 1987). 

At M-DCQ however, results indicated that while different types of students may 
indeed enroll at different times, they do not give different ratings. Statistical significance was 
not obtained on any factor for any group (see Table 13). 

Do Ratings Change Based on Instructor Rank? 

Rank tends to be viewed as a shorthand indicator of age of the faculty member and 
years of experience, though Feldman (1983) warns that age, rank, and instructional 
experience should not be thought of as interchangeable. He found that higher evaluations 
were associated with higher rank, younger teachers, and less overall experience. Marsh 
(1982) measured of experience in another way, finding that for the same instructor teaching 
the same course, higher ratings were obtained when the course had previously been taught 
at least once. Miller (1987) noted that faculty members typically receive their best ratings 
between their third and twelfth years of teaching. In about half to two-thirds of the studies, 
however, no relationship was found (Feldman, 1983). In others, higher rank was associated 
with higher ratings (Arubayi, 1987). 
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At Miami-Dade, sigiuficam differences in student ratings by rank of the instructor 
were found on four factors: (1) Focus on the Individual, (3) Approach to the Material (5) 
Usiening to Students, and (8) Active Uaming. Educational significance was also achieved. 
The strongest relationship was for the first factor where rank explained 9% of tte ratings 
variabiUty; the correlation was .29. In ^ncral, higher ratings were found for instructors and 
assistant professors compared to associate professors, associate professors senior, and full 
professors. Full professors received the least positive ratings in all cases. Further details 
can be found in Table 14. 

It is unclear exactly what these results mean. The most straightforward interpretation 
is that the longer a faculty member remains at M-DCC, tte lower the student ratings. One 
alternative explanation, however, is that some departments have more new faculty than 
others and these departments have courses that students respond more positive^ to. It is 
also possible that new faculty are not new to teaching and bring with them a variety of 
instructional experiences. These findings, however, arc not contradictory with prior research. 

Rating Chan^ Based on Subject Mattgr? 
To conduct this analysis, only departments who had at least 20 sections evaluated 
were included This reduced the number of sections from 640 to 309. On the "hard 
sciences" side, this included Applied Biology, Chemistry, Mathematics, and Nursing. The 
remaining courses were English, English-as-a-Sccond-Language, Humanities, and Psychology. 

Both statistical and educational significance were obtained for three factors: (I) 
Focus on the Individual, (7) Fairness of Examinations, and (8) Active Learning. The 
strongest relationship between subject matter and student ratings were found for Factors 1 
and a For Factor 1, subject matter explained 19% of the variability in ratings and 
correlated .44. For Factor 8, the proponion of variability explained was .16 and the 
correlation was .40. Subject matter explained about 8% of the variability for Factor 7. See 
Table 15 foi further details. 
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For Factor 1 (Focus on the Individual), subject areas receiving the most positive 
ratings were English as a Second Language* English, and Nursing. These ratings can be 
compared to the group of lowest rated subjects: Mathematics, Applied Biology, Humanities, 
Crtcmistry, and Psychology. The lowest ratings were obtained for Applied Biology while the 
highest were found for English-as-a-Second-Language courses. 

For Factor 7 (Fairness of Examinations), the only signi^'-ant difference was between 
the higher ratings in Mathematics compared to English. This is not surprising given the 
content of each area and the ways students arc tested (essay vs. problem completion or 
multiple-choice). 

For Factor 8, students saw less active learning taking place in Applied Biology, 
Humanities, and Psychology compared to Chemistry, English, Mathematics, Nursing, and 
English-as-a-Second-Language courses. Again, the lowest ratings were obtained for Applied 
Biology while the highest were found for English-as-a-Second-Language courses. 

Other researchers have found that mathematics and natural science courses were 
rated lower than humanities and social sciences (e.g., Beatty & Zahn, 1990; Braskamp et al., 
1984; Fcldman. 1978; Marsh 1984). Miami-Dade's results, however, differed from this 
pattern. Humanities and Psychology (both required core courses in most cases) received 
lower ratings than Nursing on two specific factors-Focus on the Individual and Active 
Learning. Perhaps, as Marsh (1980) found, we are seeing the confounding effects of prior 
interest intermingled with approach to the subject 

What Do Studems Think of the Survey Process? 

The last two items on the survey asked students about the length of the survey and 
the difficulty of the survey. About two-thirds of the students thought the length of the survey 
was about right. About 30% thought it was too long. Almost everyone (98%) agreed that 
the questionnaire was easy. (See Table 16). 



In addition, students were offsied the opportunity to make suggestions about the 
questionnaire. A sample of those with comments was selected for analysis of the content 
A majority of the comments from students were of the type that indicated they were 
generally satisfied with the survey. Those who had comments suggesting changes were most 
likely to comment on the length of the questionnaire or feel that the questionnaire asked too 
many questions about the respondents. A listing of the sample of comments can be found 
in Table 17. 

gtimmary and Discussion 

The first full pilot of the Student Feedback Survey, initiated in the Fall of 1990, 
invoKed randomly selecting one section to be surveyed for each full-time faculty member 
at tiie College. Surveys were returned from 640 sections (or 84% of those initially selected) 
and included 12,729 responses. The first 23 items of the survey asked students to evaluate 
the instructor and the course. The remaining 20 items requested information about the 
student 

Results showed that students were generally pleased with their classroom experiences. 
At least 95% of students thought their instructor was prepared for class, showed an interest 
in and knowledge of the subject and treated them with respect Their instructor distributed 
the course objectives and discussed the grading system with them. The only item where 
students gave significantly lower ratings was in providing regular information about their 
progress; only 62% of students agreed or strongly agreed with this item. 

Students described themselves as generally hard-working and dedicated to class 
attendance. A majority (69%) thought their performance was either "good" or "excellent". 
Most (75%) were working at least part-time and 30% indicated they had family commit- 
ments that interfered with school. They generally approved of the survey process. Based 
on headcount full-time students were over-represented by the survey, probably because they 
were more likely to be in multiple classes selected for inclusion. 
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About half described the class being evaluated as about as difficult and requiring 
similar amounts of work as their other courses. Mc»t (80%) thou^t about the right number 
of students wre in their class. A majority (71%) were in tte course because it was 
required. 

Using factor analysis, it was found that students tended to group items along eight 
dimensions (or factors). The factors were named (1) Focus on the Individual, (2) 
Competence in the Classroom, (3| Approach to the Material, (4) Grading Pohcy, (5) 
Listening to Students, (6) Clarity of Course Objectives, (7) Fairness of Ejtaminations, and 
(8) Active Learning. Thus, for example, a faculty member might be rated more highly on 
Competence in the Qassroora (Factor 2) and less highly on use of Active Learning (Factor 
8) or Fairness of Examinations (Factor 7). 

Instructors who were generally rated highty in all areas were most likely to be 
described as creating a classroom atmosphere that encoura^d learning (item 14), being 
concerned with students* progress (item 5), and making the course interesting (item 7). In 
addition, these instructors taught what they said they were going to (item 4), graded their 
exams fairly (item 11), and paid attention to student comments (item 20) according to 
student perceptions. 

The reliability for the survey was very high - .94 for the 23 items used to rate 
instructors and courses. The reliabilities for the eight factors that make up the 23-item part 
of the survey were also fairly high. They ranged from a low of .66 for Factor 2 to a high of 
.84 for Factor 3. 

Besides the issue of whether students could be reliable and discriminating judges, a 
second issue was whether ratings would change based on characteristics not directly related 
to teaching excellence. For this study, the following variables were studied at the student 
level: 'student perceptions of their classroom performance, course difBcuIty, course 
workload, class size, and reasons for taking the course. At the course level, analyses were 
conducted looking for rating changes based on time of day the course was taught, instructor 
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the end tjf the semester, and subject tau^t Dififerences large enough to be labeled 
"educationally significant" were found for three variables: student perceptions of their 
classroom performance, instructor rank, Ind subject matter. 

On the issue of student classroom performance, results showed that students who 
rated their performance as "good" or "eiscellent" gave higher ratings on aU eight fertors than 
students who did not feel they were doing as well in dass. This may mean that those who 
arc doing well in class may find the instructor more ^ctivc than ihost who feel they are 
not understanding the material Another explanation is that tl» simple expectation of a hi^ 
grade will produce higher evaluations. In general, past research has indicated that amount 
of learning rather than grades is a better indicator of mstructor ratings. 

Differences in ratings were also found based on professional rank for four of the 
factors: (1) Focus on the Individual, (3) Approach to the Material, (5) listening to 
Students, and (8) Active Uaming. Instructors and assistant professois obtained higher 
scores than faculty at other ranks. Again, several interpretations are possibte. One is that 
those who are "freshest" to M-E>CC cany that freshness into the classroom. Prior research 
bears this interpretation out Miller (1987), for example, found that faculty receive their 
highest ratings between their 3rd and 12th years of teaching. Another possibiUty is that 
another variable related to faculty rank is actually responsible for the difference. For 
example, some departments or subject areas which students enjoy more may have a greater 
proportion of new faculty and it is the subject area causing the differences. 

Indeed, differences based on subject matter did emerge from the analyses on three 
factors. Students enrolled in English-as-a-Second-Language courses, English, and Nursing 
courses provided higher ratings in the area of Focus on the Individual than Mathematics, 
Applied Biology, Humanities, Chemistry, and Psychology. In the area of Active Learning 
(Factor 8), students rated Chemistiy, English, Mathematics, Nursing, and English-as-a- 
Second-Lan^age courses as higher than Applied Biology, Humanities, and Psychology. 
Finally, students perceived Mathematics examinations as fairer than English examinations. 
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White some of the differences must surely be due to the content (e.g., hard science vs. the 
humanities), others are not as readily explained 

Statistical significance was found for some other variables, but the mean differences 
were not as great as for those already mentioned. These include course difficulty, amount 
of work required, class size, and reasons for taking the course. There were no differences 
when ratings were viewed based on the time of day the course was offered. 

Taken as a whole, these results seem to confirm that the Student Feedback Survey 
is basically a reliable and valid instrument The next series of anafyses using the Winter, 
1991, data will look at the variables in combination instead of one at a time, study the effects 
of these variables when the effectiveness of the instructor is held constant (by teaching 
multiple sections of the same course), and seek to confirm the factor analytic pattern found 
in the Fall data. 
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Results of Fall 1990 Pilot of Student feedback Survey: 
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Strongty Disagree (4) 


19J 


4.7 


316 


5.6 


29 


2.2 


49 


3.8 


14 


7.9 


601 


4.8 


total 


4.069 


100.0 


5.709 


100.0 


1.319 


100.0 


1.277 


100.0 


178 


100.0 


12.552 


100.0 



Hissing 3 \JJ 



I 
I 



Tlie Instructor Is HvoHaMe For Individtet ielp 



Strongly Agree (1) 


2.063 


50.7 


2.678 


47.0 


771 


58.4 


653 


51.4 


96 


53.9 


6.261 


50.0 


Agree (2) 


1.594 


39.2 


2,426 


42.6 


452 


34.2 


475 


37.4 


66 


37.1 


5.013 


40.0 


Disagree (3) 


328 


8.1 


455 


8.0 


SO 


6.1 


114 


8.9 


12 


6.7 


989 


7.9 


Strongly Disagree (4) 


82 


2.0 


136 


2.4 


18 


1.5 


29 


2.3 


4 


2.3 


269 


2.1 


Total 


4.067 


100.0 


5.695 


100.0 


1,321 


1(K).0 


1.271 


100.0 


178 


100.0 


12,532 


100.0 


Missing = 197 
































9. Ihe 


Inatructer Ei 




1 Quest fern 


In Ct«M 












Strongly Agree (1) 


2,336 


57.4 


3.025 


52.8 


838 


63.2 


700 


54.6 


102 


57.0 


7,001 


55.6 


Agree (2> 


1.412 


34.7 


2,053 


35.8 


411 


31.0 


460 


35.9 


60 


33.5 


4.596 


54.9 


Disagree {3) 


271 


6.6 


513 


9.0 


67 


5.1 


94 


7.3 


11 


6.2 


956 


7.6 


Strongty Disagree (4) 


54 


1.3 


157 


2.4 


9 


0.7 


2t 


2.2 


6 


3.5 


234 


1.9 


lot at 


4.073 


100.0 


5.728 


100.0 


1.325 


100.0 


1.282 


100.0 


179 


100.0 


12.587 


100.0 


Ni!ising 2 142 



























.«<;) 



3(; 



TftbU t 

Ccontinued) 



Resuitft of Fall 1990 FiUt of Student Fooc».:k Survoyt 
ItoM Rotated to focuUif Chtrocterittfcf 



Coitus 



1 


North 


South 


ttotfson 


Nedicat Canter 


Honetcead 


CoUaea-yide 




Ititfter 


Forcent 


ttu*er 


Percent 


NiMlier 


Ptrctnt 




Farcent 


Nuiter 


Percent 




Percent 




10. lim CMvfnotlon 


• Antf/Or OttMT For 


Off eMtiatfon *n» lelatod To Ik* tturm Nitarfat 








Strongly Agree (1) 


2,641 


64.9 


5.705 


64.8 


865 


6S.2 


722 


56.9 


109 


61.6 


8.040 


64.0 


Agree (2) 


1,251 


30.8 


1.741 


30.4 


408 


50.8 


451 


54.0 


61 


54.S 


5.092 


51.0 


DIsegree C5> 


159 


J.4 


211 


5.7 


47 


5.5 


81 


6.S 


5 


2.8 


485 


3.9 


Strongly Disagree (4) 


37 


0.9 


64 


1.1 


7 


0.5 


53 


2.6 


2 


1.1 


145 


1.1 


Total 


4,068 


100.0 


5.719 


100.0 


1,527 


100.0 


1.269 


100.0 


177 


1.4 


12.560 


100.0 


Mis&ing = 169 






























11. The f» 


■iifwiio 


m And^Or Other Form 


Of Evatuotion Are Graded Foirly 










Strongly Agree (!) 


2,429 


60.0 


5,311 


57.9 


811 


61.5 


706 


55.5 


102 


57.5 


7.559 


58.7 


Agree (2| 




35. 0 


1,942 


54,0 


429 


52.4 


445 


55.0 


65 


55.4 


4.214 


55.6 


Disagree (3) 


225 


5.5 


352 


6.2 


71 


5.4 


95 


7.5 


11 


6.2 


752 


6.0 


Strongiy Disagree (A> 


65 


1.5 


111 


1.9 


12 


0.9 


26 


2.0 


2 


1.1 


214 


1.7 


total 


4,0S0 


100.0 


5.716 


100.0 


1.525 


100.0 


1.272 


100.0 


178 


100.0 


12.559 


100.0 


Hissing = 190 






























12 


. Ihe Irntnjctor Itode The Sradine SyatcB 


Clear To He 










Strongly Agree (1> 


2.381 


SB. 4 


5.158 


55.2 


817 


61.8 


765 


59.9 


85 


47.5 


7.204 


57.5 


Agree (2) 


1 J31 


52. f 


2.004 


55.0 


422 


52.0 


429 


55.7 


72 


40.2 


4,258 


55.9 


Disagree (5) 




7.4 


445 


7.8 


71 


5.4 


70 


5.5 


19 


10.6 


903 


7.2 


Strongly Disagree (4) 


61 


1.5 


114 


2.0 


11 


0.8 


11 


0 9 


5 


1.7 


200 


1.6 


fotal 


4,073 


100.0 


5,719 


100.0 


1.521 


100.0 


1,275 


100.0 




100.0 


12.565 


100.0 



Kissing 164 



:?7 



ERIC 
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Tabte 1 

(continued) 



Results of rail 1990 Pilot of Student Feedtock Sirvay: 
ttoK Retated to Faculty Cttaracttri sties 



» 


MOrth 


south 


Uolfson 


Mdicat 


Center 




Cetlega-Uids 






Percent 


Nuiter 


Percent 




Parcent 


Niater 


Parcent 


Nudier 


Parcent 


Niater 


Percent 








IS. The Instructor FraMnts D 


Burw Nitcrial Claarly 










Strongly Agree <1> 


2,233 


54.8 


2,834 


49.5 


826 


62.5 


651 


51.1 


76 


43.2 


6.620 


52.7 


*free (2) 


1,442 


35.4 


2.140 


37.4 


408 


30.9 


452 


35.4 


68 


38.6 


4.510 


35.9 


Disagree (3) 


315 


7.7 


571 


10.1 


69 


5.2 


132 


10.4 


26 


14.8 


1.113 


8.9 


Strongly Disagree (^) 


83 


2.1 


174 


3.0 


18 


1.4 


40 


3.1 


6 


3.4 


321 


2.5 


total 


4,073 


100.0 


5,719 


100.0 


1,521 


1(»).0 


1,275 


100.0 


176 


100.0 


12.564 


100.0 


Nis&tng s 16S 




























14. fhe Instructor Creates A Clasi 


irooB ktmatftien Tliat Encouraec* *te to Leom 








Strongly Agree (1) 


2,060 


50.6 


2,557 


44.8 


789 


59.6 


651 


51.2 


63 


35,2 


6,120 


48.8 


Agree (2> 


1,485 


36.5 


2,192 


38.4 


420 


31.8 


438 


34.4 


84 


46.9 


4.619 


36.S 


Disagree C3) 


395 


9.7 


708 


12.4 


88 


6,6 


147 


11.6 


26 


14.5 


1,362 


10.8 


Strongly Disagree (4) 


150 


3.2 


250 


4.4 


26 


2.0 


36 


2.8 


6 


3.4 


448 


3.6 


Total 


4,068 


100,0 


5,707 


100.0 


1,323 


IW.O 


1,272 


100.0 


179 


100.0 


12,549 


100.0 


Missing > 180 
































15. The Instructor Dcmstrates Kmwledie Of Ihm Si^lecK 










Strongly Agree (1) 


2,925 


71.8 


4,158 


72.8 


978 


74.0 


874 


68.7 


11.9 


66.5 


9.052 


72.1 


Agree (2) 


1.015 


24.9 


1.581 


24.2 


31S 


23.7 




28.3 


57 


31.8 


3,124 


24.9 


Disagree (3) 


109 


2.7 


134 


2.4 


22 


1.7 


30 


2.3 


2 


1.1 


297 


2.4 


Strongly Disagree (4) 


27 


0.6 


38 


0.7 


8 


0.6 


9 


O.T 


1 


0.6 


85 


0.6 


Total 


4,072 


100,0 


5.711 


100.0 


1,321 


100.0 


1,273 


im).o 


179 


100.0 


12.556 


100.0 


Mtssing s 173 



























I 

lO 

» 



ERIC 



Aao42\V' 



4U 



Table 1 

(continued) 



ftesuiu of fmii 1990 Pilot of Student Feedbock Swv^i 
licw Retoied to Foctilty Character Ut In 



Itorth South liolfBon iMical Center ttoMstead Colle9a*yi<ia 



auiber Percent tfiaber Percent IMber Peremt Ntiiber Percent mmimr Percent Kuiter Percent 



16* The tnetrvcfor Umo A Variety Of Taacftini Netboda 
(for Eaa^ple^, lecture, Diacuaalona, taonatratiena^ AiidioviaiAl lllda imdfOt Otbera) 



Strongly Agree (1) 


2.091 


51.4 


2,538 


44.5 


732 


55.5 


649 


51.3 


71 


59.7 


6.081 


48.5 


Agree (2) 


1,359 


33.4 


2,087 


36.6 


423 


32.1 


442 


34.9 


75 


41.9 


4.386 


35.0 


Disagree {}) 


5t0 


12.6 


826 


14.5 


133 


10.1 


14$ 


11.4 


24 


15.4 


1.638 


15.1 


Strongly Disagree ik) 


107 


2.6 


251 


4.4 


50 


2.3 


30 


2.4 


9 


510 


427 


5.4 


Totat 


4,067 


100.0 


5,702 


100.0 


1.318 


100.0 


1.266 


100.0 


179 


100.0 


12.532 


100.0 


Missing > \97 


























17. Aaalgrwcnta itetp Ne team The Ceurae Naterial 


Strongly Agree (1) 


2,011 


49.6 


2,405 


42.3 


710 


53.6 


539 


42.4 


84 


46.9 


5.749 


45.9 


Agree (2) 


1.605 


39.6 


2,406 


42.3 


477 


36.0 


sss 


43.8 


75 


40.8 


5.119 


40.9 


Diikagree (3) 


555 


8.7 


666 


11.7 


117 


8.8 


135 


10.6 


14 


7.8 


1.287 


10.3 


Strongly Disagree (4) 


86 


2.1 


215 


3.7 


21 


1.6 


41 


3.2 


B 


4.5 


370 


2.9 


Total 


4.057 


100.0 


5,691 


100.0 


1,325 


100.0 


1.273 


100.0 


179 


100.0 


12.525 


100.0 


Missing » 204 
































IB. The Instructor Encourages He To Thfnfc For %Mlf 












Strongly Agree (1) 


1.985 


48. S 


2.472 


43.4 


735 


55.7 


fr30 


49.5 


78 


43.8 


5.900 


47.1 


Agree (2) 


1.680 


41.3 


2,490 


43.8 


467 


35.4 


S13 


40.3 


S9 


50.0 


5.239 


41.8 


Disagree (5) 


S2t 


7.9 


587 


10.3 


94 


7.1 


111 


8.7 


10 


5.6 


1.125 


9.0 


Strongly Disagree (4) 


81 


2.0 


144 


2.5 


24 


1.8 


19 


1.5 


1 


0.6 


269 


2.1 


Totat 


4.067 


100.0 


5,693 


100.0 


1,320 


100.0 


1,273 


iro.o 


178 


100.0 


12,531 


100.0 


Missing = 198 
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Table 1 
Ccontinued) 



fkesults of FftU 1990 Ptlot of Student fotdbock Sirvoy: 
Item Related to Fociilty OiorKtorUtks 



CoMpus 



north SoMtti Hot f son Ncdicel Center Howsteed Col I tot-Wide 



Nuater Percent Umter Percent Nuiter Percent tuiiier Percent Huiter Percent U%mb$r Percent 



19. Ttie instructor Infom We iegulerly Hbout Ny Progre— 



Strongly Agree (1> 


t.072 


26.S 


1.115 


19.7 


495 


57.4 


390 


M.7 


39 


21 .B 


3,109 


25.0 


Agree (2) 




58.5 


1.980 


55.1 


491 


57.2 


455 


35.9 


70 


39.1 


4,551 


36.6 


Diiiagree |5) 


1,113 


27.6 


1.87B 


55.5 


279 


21.2 


527 


25.8 


56 


31.3 


3.653 


29.3 


Strongly Disagree (4) 


298 


7.4 


675 


11.9 


55 


4.2 


96 


7.6 


14 


7.8 


1,136 


9.1 


Total 


4,038 


100.0 


5,646 


fOO.O 


1,518 


100.0 


1,268 


100.0 


179 


100.0 


12,449 


100.0 


HI string - ?80 

_ — „ 


■ 































20. The 


iratructor 


Pays Attention fo Hy Umment* 













Strongly Agree (1) 


2.101 


51.6 


2.591 


45.4 


767 


58.5 


627 


49.4 


85 


47.7 


6,171 


49.2 


Agree {2} 


1,618 


59.8 


2.529 


44.4 


465 


35.5 


512 


40.5 


72 


40.5 


5,196 


41.5 


M filagree |>/ 




A ft 


•»? 1 


.'.9 


67 


5.1 


96 


T.o 


i6 






7.2 


Strongly Disagree (4> 




t.a 


131 


2.5 


17 


1.5 


34 


2.7 


s 


2.8 


261 


2.1 


Toial 


4.069 


100.0 


5,702 


100.0 


1.516 


100.0 


1,269 


100.0 


178 


100.0 


12,554 


100.0 


Hissing = 195 
































21. 


Ihe Instructor Ircst* He Itith ictpect 












Strongly Agree (1) 


2.699 


66. S 


5.507 


61.4 


999 


75.5 


787 


61.7 


100 


55.9 


8.092 


64.5 


Agree (2) 


1.168 


28.7 


1,895 


55.2 


274 


20.7 


418 


32.8 


66 


56.8 


5,821 


30.4 


Disagree <3) 


155 


5.5 


205 


5.5 


55 


2.6 


49 


3.8 


8 


4.5 


428 


5.4 


Strongly Disagree (4) 


61 


1.5 


107 


1.9 


16 


1.2 


22 


1.7 


5 


2.8 


211 


1.7 


Total 


4.061 


100.0 


5,712 


100.0 


1,524 


100.0 


1,276 


100.0 


179 


100.0 


12.55> 


100.0 



Hissing - \77 



AII042.1 



BEST COPY AV/ULABLE 



Tabu 1 
Ccontinged) 



Results pf f^^l 19W Pilot of Student Feedbock $urv«y: 
Item Relftted le Faculty Oiarieterfstics 



Ci^pus 





North 


South 


Wolf son 


Nedicel Center 


Kowstead 


Colter 


i-Vtde 


tkmber 


Percent 


IhJitier 


Percent 


Umter 


Percent 


Ififliier 


Percent 


Ninber 


Percent 




Percent 




22. 


Tlie liwtructor DiscuHad Ttm firadfnB Sy«tcB At The fegimlng Of flM Sm 


Mter 








strongly Aaree (1) 


2,687 


66.1 


3,561 


62.3 


916 


69.2 


839 


66.1 


105 


58.7 


8.108 


64.6 


<l9ree (2) 




28.4 


1,824 


31.9 


354 


26.7 


379 


29.9 


62 


34.6 


3.773 


30.1 


Pisagree (3) 


t74 


4.3 


274 


4.8 


46 


3.5 


41 


3.2 


9 


S.O 


SU 


4.S 


Strongly Disagree {4> 


48 


1.2 


58 


1.0 


8 


0.6 


10 


0.8 


S 


1.7 


127 


1.0 


Total 


4,063 


100.0 


5.717 


100.0 


1,324 


100.0 


1,269 


100.0 


179 


100.0 


12.552 


100. 0 


Nissing = 177 
































23. 


The instructor Starts Class 


on Tiae 












Strongly Agree (1> 


2,5SJ 


62.8 


3.677 


64.2 


942 


71.0 


841 


66.3 


126 


70.4 


8,139 


64.7 


Agree (2) 


t.171 


28.8 


1.692 


29.6 


319 


24.1 


355 


28.0 


42 


23.4 


3,579 


28.5 


Disagree (3) 


250 


6.2 


281 


4.9 


55 


4.1 


$7 


4.5 


6 


4.5 


651 


5.2 


Strongly Disagree (4> 


90 


2.2 


77 


1.3 


10 


0.8 


IS 


1.2 


5 


1.7 


195 


1.6 


total 


4.064 


100.0 


5,727 


100.0 


1,326 


100.0 


1.268 


Ittt.O 


179 


10O.0 


12,564 


100.0 



t 

I 



Missing ^ 165 



if. -u; 



er|c 









Man 




north eiiviit 



Xt«B 

1 

2 
3 
4 
9 
< 
7 
B 
9 

10 
11 
12 
13 
1« 
IS 
IC 
17 
18 
19 
20 
21 
23 
33 



ractor 
1 
2 
3 
4 
5 
( 
7 
S 

Total 



XMtmeter i« Fr«p*r«d for cImj 
XMtntetor iImm lBt«r»«t la subjoct 
Diitril»>t«4 coar»» <*J«etivjj 

ceae«rs«d Nltli iqr rrofr*** 

sbawa Bow tutorial Can Booofit 

MakM coorao xstoroatlnQ 

Availablo for Xndlvidaal Bolp 

Boeoorama gaoacioita in data 

BvalaatiOB Rolatod to Natorial 

Exaas «rad«d Fairly 

orading fyataa doar 

PraioBtt Katorial eloarly 

craatoa AtaeapbMr* Encouraging Loarning 

Doaonatrataa Rsowlddgo of sobjoet 

Oaaa varioty of Taaehing Nothoda 

AaaignawBta Bolp x«araing 

BMOoragoB TDiiucing for Solf 

Xnfom Rogularly Progra«a 

Faya Attontieo to eaaaBta 

nraata mm With Raa^t 

Oiieuaaod Grading at Baginning 

Starts elaaa on Tiaa 



reeua on individual ( 9.19.6,8) 
CoBpotanca in claaaroom c2,l,15> 
Approach to Hatarial (14,9.13,16) 
orading Policy (22,12) 
Liataning to Stadanta (30,21.9) 
Clarity on ceuraa <Ajaetivaa (4.3) 
raimaaa of Exawa (ll.ie.l2*» 
Activa Laarning (17,18) 



4.078 
4,080 
4.073 
4,098 
4,061 
4.086 
4.069 
4,067 
4.073 
4,068 
4,050 
4,073 
4,073 
4,068 
4,072 
4,067 
4,097 
4,067 
4,038 
4,069 
4,061 
4,063 
4,064 



4,090 
4,090 
4.087 
4,089 
4,087 
4,084 
4,087 
4,082 

4.091 



1.30 


0.53 


1.29 


0.48 


1.46 


0.64 


1.58 


0.68 


1.76 


0.80 


1.72 


0.79 


1.71 


0.85 


1.61 


0.72 


1.92 


0.68 


1.40 


0.60 


1.49 


0.67 


1.52 


0.70 


1.57 


0.72 


1.65 


0,78 


1.32 


0.56 


1.66 


0.79 


1.63 


0.73 


1.63 


0.71 


2.16 


0.90 


1.59 


0.70 


1.40 


0.63 


1.41 


0.63 


1.48 


0.71 



7.19 


2.48 


3.85 


1.29 


6.57 


2.54 


2.91 


1.17 


4.48 


1.63 


3.03 


1.17 


4.38 


1.60 


3.25 


1.25 


35.60 


10.36 



South CABSpttS 



1 XMtruetor i« Pr»par«a for 

2 In»tructor Showi lBt*r««t in subject 

3 Di»trib«t«d couria obj#ctiw« 

4 obJ«€tiv«» And what !• T»ufht Agr«e 

5 concmtnmd with my Progr»«« 

6 shawm tlov lutsrial Can B«na£it 

7 futos cotxrmm Xntar*ttin9 

8 AvailAbla for xndividuai MXp 

9 CTCourAaos Qutvtiens in CIms 

10 EVAluAtieft MlAtad to NAtarial 

11 zx$mm or mdmd Fairly 

12 Grading Systm claar 

X3 Praaonta Hatarial Claar ly 

14 craataa Atmoaphara Encouraging Laarning 

15 Daaonatrataa Knowladga o£ subjact 

16 Oaaa variaty of Wachlng Mathoda 

17 Aaaignsottta 9alp taaming 

IS Encouragaa Thinking for salf 

19 xnforM Ragularly prograaa 

20 Pay a Attantion to Cemmanta 

21 Traata »• with Raapact 

22 Diaeuaaad Grading at Baginning 

23 starta Claaa on Tiina 



5,735 


1.32 


0.54 


5,743 


1.27 


0.52 


5,724 


1.47 


0.64 


5,709 
5.492 


1.57 


0.69 


1.93 


0.83 


5,700 


1.S4 


0.84 


5,709 


1.80 


0.87 


5,695 


1.44 


0.73 


5,728 


1.41 


0.75 


5,7X9 


1.41 


0.62 


5,714 


1.52 


0.70 


5,719 


1*57 


0.72 


5,719 


1.44 


0.78 


5,707 


1.74 


0.83 


5,711 


1.31 


0.55 


5,702 


1.79 


0.85 


5,491 


1.77 


0.80 


5,493 


1^72 


0.75 


5,444 


2.37 


0.93 


5t702 


1.47 


0.72 


5,712 


1.46 


0.46 


5,717 


1.45 


0.44 


5,727 


1.43 


0.65 



Factor roc\xm on individual (5,19,4,8) 

2 Compatanea in claaaroom <2,1,15) 

3 Approach to Hatarial cl4,9,lJ»16) 

4 orading Policy (22,12) 

. 5 Liataning to Studanta (20,21,9) 

4 i Clarity on court a Objactivaa (4,3) 

7 rairnaaa of Exama (11,10,12*) 

8 Active Laarning (17,18) 

Total 



5,746 


7.73 


2.40 


5,750 


3.88 


1.31 


5,747 


6.98 


2.71 


5,743 


3.00 


1.20 


5,745 


4.71 


1.77 


5,741 


4.48 


1.65 


5,747 


3.03 


1.18 


5,731 


3.47 


1.65 


5,750 


37.11 


10.73 



•Includad in mora than ona factor. 
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MMB Mault* of rail lf»» rtlot of ttadMit y—ifeack tor^y 



DOVlatlOR 



Zt«a 

1 
3 
9 
4 

9 
« 

7 
8 
9 

10 
11 
12 
13 
14 
IS 
16 
17 
18 
19 
20 
21 
22 
23 

raeter 
1 
2 
3 
4 
S 
6 
7 
8 

Total 



Factor 
1 
2 
3 

- 4- 
5/ 
S 
7 
9 

Total 



SBftnietor it rroparod for clasa lr3M 

xnatmetor tkowa maraat ia fubjact 1.330 

Piatcitatad eooraa otejacUvaa 1.324 

^jaetim and nhat ia Taught *flraa l,32i 

eeaeantad with mf rro«raaa i.«0 

SlMwa BOW Natariai caa lamfit 1,321 

nakaa Miuraa maraatlM t'nX\ 

Availabta for Zndividoal Salp 1.321 

Bneouragaa gooatieiia in claaa t'iiz 

Bvalttatien Ralatad to Natariai 1,327 

Bxaaa oraM Fairly }'373 

aradlB9 syataa cUar J*"* 

rroMnta Matarial claariy J, 321 

craataa Ataeaphara meooragiaa taaminf t'itl 

oaaonatrataa Knewladga of tuJ»foet 1.321 

Oaaa variaty of taaebinv Mtbeda 1. 3 if 

AaaiQBBaata 8alp laaminf l.3» 

BBeettra«aa Thinking for aalf l."0 

Znfexva Mgolarly rrograaa 1.318 

paya AttantlM to cemanta t'lW 

Traaea aa Kith Mapaet J."* 

Olaeuaaad Oradiag at tagimiiag 1.324 

Starta cXaaa on Ti»a i.^aa 



roeua oB Xndividual (9,19,6,8) 
Coapotanco in Ciaaaroon (2,1,19) 
Approach to Matarial (14,9,13,18) 
orading Poliey (32.12) 
Liataoiag to atudanta (20,21,9) 
clarity en Ceuraa objoetivva (4,3) 
rairnaaa of Cxana (11,10,12*) 
Aetiv* Laaming (17,18) 



1,329 
1,330 
1,327 
1,325 
1,328 
1,328 
1,329 
1,336 

1,330 



1.3C 

1.24 

1.41 

l.SO 

1.60 

1.62 

1.91 

1.90 

1.43 

1.39 

1.4« 

1.45 

1.45 

1.51 

1.29 

1.59 

1.5S 

1.95 

1.93 

1.49 

1.30 

1.35 

1.35 



6.60 
3.78 
6.03 
2.80 
4.30 
3.90 
4.39 
3.13 

33.6 



Nodical cantar canpua 



ItaiB 

I tnatructor la Praparad for Claaa 

3 Xnatruetor Showa Xntaratt in subject 

3 oiatribtttad ceuraa objaetivaa 

4 Objactivoa and what ia Taught Agrao 
9 coBcamad Nlth ry Progroaa 

8 Sbeva How Matarial can Sanafit 

7 Nakaa Ceuraa ZBtaraating 

8 Available for individual aalp 

9 Bneouragaa ouaatlona in Claaa 

10 Evaluation Bolatad to Natariai 

11 Exasa Oradad raixly 

12 arading Syaten Clear 

13 Praaanta Material Clearly 

14 createa Ataoapbare Encouraging Learning 

15 oeaoRatratea Knowledge of Subject 

16 Oaea variety of Teaching Metboda 

17 Aaaignaanta Belp trfiamiBg 

18 Encouragaa Thinking for self 

19 xnforaa Regularly Prograaa 

30 Paya Attention to Co»»»nta 

31 Treata am with Reepect 

33 Oiaeuaied Grading at Beginning 

23 Starta Claaa on Tiae 



Foeua on individual (5,19,6.8) 
Competence in claatroom (2,1,19) 
Approach to Material (14,9,13,16) 
Grading Policy (22.12) 
Liatening to Studenta (20.21,9) 
Clarity on Courae objectivaa (4,3) 
rairneaa of Exana (11,10,12*) 
Active Learning (17,18) 



•Included m more than one factor. 
AB042.5 
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0.50 
0.17 
0.61 
0.65 
0.74 
0.73 
0.73 
0.67 
0.62 
0.58 
0.64 
0.64 
0.66 
0*71 
0.52 
0v76 
0.72 
0.71 
0.66 
0.66 
0.51 
0.5f 
0.60 



2.39 
1.26 
2.37 
1.09 
1.52 
1.12 
1.54 
1.27 

10.3 



1,263 
1,279 


1.34 


0.53 


1.28 


0.49 


1,273 
1,270 


I445 


0.63 


1.60 


0.74 


1,267 


1.70 


o.ao 


1,275 


1.69 


0.78 


1,277 


1.68 


0.81 


1,271 


1.62 


0.74 


1,262 


1.57 


0.72 


1,269 


1.S5 


0.73 


1,272 
1,273 


1.56 


0.72 


1.47 


0.64 


1,275 


1.66 


0.79 


1,272 


1.66 


0.79 


1,273 


1.35 


0.56 


1,266 


1.65 


0.77 


1,273 
1,273 


1.75 


0.77 


1.62 


0.71 


1,268 


2.10 


0.93 


1,269 


1.64 


0.74 


1,276 


1.46 


0.65 


1,269 


1.39 


0.59 


1,268 


1.41 


0.63 



1.288 


7.02 


2.60 


1,291 


3.93 


1.33 


1,287 


6.57 


2.65 


1,282 


2.84 


i.ll 


1,290 


4.61 


J. 79 


1,283 


3.02 


1.21 


1,288 


4.52 


1.76 


1,283 


3.34 


1.31 


1,98 


35.48 


11.50 



{eratiiio«d) 

MM ittmiUft of t%Xl Iff 0 f tlQt oi 9Z^amn% W db ack Survy 



O#vl*tioii 



factor 

I 
2 
3 
4 
S 
€ 
7 

a 

Totftl 



rector 
1 
2 
3 
4 

5 : 
6 
7 
8 

Total 



1 sutruGtor im rroparod for ««•• 

2 XMtruotor fbow xntoroat is subjoct 

3 piitritetMl coiurM objoetlvot 

4 objoetlv«o ud wliat it Toagbt Agroo 

5 cenouMtf with mf fro^roto 

« item Bov lUtorUI Can Bonofit 

7 Mateo courao xotoroatiog 

$ Availablo for Zodividuol Bolp 

9 KBOOuroooo Qwationa in Claaa 

10 BVBluotion teUtod to Mtorial 

11 mumm ttadad rolrly 

12 oratfiov BfBtmm cloar 

13 rroiooto HBtorial clooriy 

14 croBtoa Jkt»Bptero BooottrogiBf Looming 

15 DoaniotrotoB Koovlodfo of oub^oct 
l( oaoo vorloty of Toaobing Mtboda 

17 MoifBBonta Bolp toaming 

18 noouro^o Tbiuing for soif 

19 Znforva Rogolariy Progrooa 

20 Foyo Mtootion to cooMnta 

21 troota m tritb Boopoot 

22 Piaeuaaod (nroding at Boginning 

23 itarta Claaa on Tim 



180 


l.«2 


0.62 


178 


1.21 


0.42 


178 


1.41 


0.62 


178 


l.ftfi 


0.71 


178 


1.74 


0.77 


17? 


1.90 


0.77 


178 


1.98 


0.93 


178 


1.57 


0.72 


179 


1.55 


0.76 


177 


1.43 


0.61 


178 


1.51 


0.§7 


179 


l.ftC 


0,73 


17ft 


1.78 


0.82 


179 


1.86 


0.79 


179 


1.3S 


0*54 


179 


1.84 


0.84 


179 


1.70 


0.80 


178 


l.fiS 


0.62 


179 


2.35 


0,89 


178 


l.«7 


0.76 


179 


1.54 


0.71 


179 


l.SO 


0.67 


179 


1.J7 


Q.6S 



fooua on Individual c 9, 19,6,8) 
cosffiotaneo in claaarom {2«1,15) 
Approocb to Matorial (14,9,13,16) 
Qrading Policy (33,12) 
X^iatoning to studonta (20,21,9) 
clarity on courao objactivoa (4,3) 
raimoaa of exam (11^10,12^) 
Activo Looming (17^ 18) 



179 


7.44 


2.37 


IBO 


3.97 


1.28 


179 


7.42 


2.73 


179 


3.16 


1.38 


179 


4.76 


1.94 


179 


3.05 


1.19 


179 


4.59 


1.61 


179 


3.32 


1.37 


180 


37.23 


11.16 



Colloga-nida 



1 tnatructor ia Proparod for claaa 

2 xnatructor showa Xntoraat in subjact 

3 Piatritetad courao objactivaa 

4 (Ajoctivoa and what ia Taught Agraa 

5 coneomod witb wy Progroaa 

6 showa BOW Mtorial can Bonof it 

7 MaJfoa courao xntoraating 

8 jivailablo for individual Bolp 

9 Bnoouragoa ouoationa in claaa 

10 evaluation Bolatod to Matorial 

11 EBama orated fairly 

12 orading syaton cloar 

13 Proaonto Matorial cloar ly 

14 Croatoa Atnoapboro Bneeuraging Laarning 

15 paaonatratoa Knowladgo of subjoct 

16 Paoa variaty of Toacbing Mathoda 

17 Aaaignnonta Bolp Looming 

18 sneouragoa TbinWng for salf 

19 Xnfor«a Bagularly Prograaa 

20 Pay a Attontion to cofl»anta 

21 Troata »o with Roapoct 

22 Diaooaaad Orading at Baginnlng 

23 starta Claaa on Tiino 



12,606 


1.31 


0.53 


12,610 


1.26 


0.50 


12,574 


1,46 


0.63 


12,636 


1.57 


0.69 


12,518 


1.82 


0.81 


13,541 


1.78 


0.81 


13,552 


1.73 


0.85 


12,532 


1.62 


0.72 


12,587 


1.57 


0.71 


12,560 


1.42 


0.62 


12,539 


1.51 


0.69 


12,565 


1*53 


0.70 


12,564 


1.61 


0.75 


12,549 


1.69 


0.80 


12,556 


1.32 


0.55 


12,532 


1.71 


0.82 


12,525 


1.70 


0.77 


12,531 


1.66 


0.73 


12.449 


2.23 


0.93 


12,534 
12,552 


1.62 


0.71 


1.42 


0.64 


12,552 


1.42 


0.63 


12,564 


1.44 


0.67 



rocua on individual (5,19,6,8) 
Cosqp«tonca in Claaaroom (2,1,15) 
Approach to Matorial (14,9,13.16) 
orading Policy (22,12) 
Liatoning to studanta (20,21,9) 
Clarity on courao objactivaa (4,3) 
rairnata of ExA»a (11,10,12*) 
Activa Laarning (17,18) 



12,632 


7.36 


2.56 


12,641 


3.86 


1.29 


12,627 


6.71 


2.63 


12,614 


2.93 


1.17 


12.629 


4.57 


1.71 


12.615 


3.01 


1.17 


12,630 


4.43 


1.64 


12.601 


3.35 


1.31 


12.649 


36.08 


10.72 



•Included in »or« than on» factor. 
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ERIC 



-32- 



•Jfl 



Table S 



Results of fmi\ 1990 Pilot of Student feedback %ury^i 
Item Seloted to Student Chorecteristics 

■ — ' 

Cwpus 



North South Wolfson Hedicel Center Honesteod CoUtge-Mide 



Utmtier l>ercent HiMber Percent Uwtmr Percent Hiatoer Percent Kuster Percent Ik^ier Percent 



24. ioit Often Po You Coae To ClOM? 





5,447 


84. S 


4,B6d 


85.2 


1,097 


83.2 


1.160 


91.5 


145 


81.9 


10,717 


85.4 


Often 


^49 


15.5 


73* 


12.8 


198 


15.0 


88 


6.9 


27 


15.3 


1.596 


12.7 


fjfliet tines 


$S 


1.4 


104 


1.8 


20 


1.5 


IS 


1.4 


S 


2.S 


202 


1.6 


Alfnost Never 


13 


0.5 


10 


0.2 


4 


0.3 


s 


0.4 


0 


• 


32 


0.3 


Total 


4.064 


100.0 


5.716 


100.0 


1,519 


100.0 


1,271 


100.0 


177 


100.0 


12.547 


100.0 


Mtssino - 182 






















— 











». 


NoM Often Are Vou Prepared For 


CtassT 


- 


- - - 


- ■ — — 








2,647 


65.0 


3,611 


63.1 


755 


57.0 


729 


57.3 


136 


76.4 


7,876 


62.7 


Often 




CO. 6 


i ,OVO 


29.7 


469 


35.5 


438 


54.4 


35 


19.7 


3.801 


30.3 


Somet tne^ 


240 


5.9 


365 


6.4 


91 


6.9 


96 


7.6 


7 


3.9 


799 


6.3 


Almost Kever 


20 


O.S 


46 


0.8 


8 


0.6 


9 


0.7 


0 




85 


0.7 


Total 


4.070 


100.0 


5,718 


100.0 


1,321 


100.0 


1.272 


100.0 


178 


100.0 


12,559 


100.0 


Missing ' 170 
































26. IH 


m Often Do Vou Pay Attenticn In CtossT 












Al{FK>st Always 


?,875 


70. a 


3,959 


69.6 


1.024 


77.6 


992 


78.2 


125 


69.9 


8,973 


71.7 


Often 


987 


24.3 


1.409 


24.8 


262 


19.9 


229 


18.0 


57 


21.0 


2.W4 


23.4 


Sofnet tmes 


160 


5.9 


254 


4.4 


25 


1.7 


37 


2.9 


15 


8.5 


489 


3.9 


Almost Never 


39 


1.0 


67 


1.2 


10 


0.8 


11 


0.9 


1 


0.6 


128 


1.0 


total 


4,061 


100.0 


5,689 


100.0 


1,319 


100.0 


1.269 


100.0 


176 


100.0 


12.514 


100.0 



Hissing = 215 
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TftbU 3 

Cconttnued) 



Results of Ft(t 1990 Pilot of Student feedbeck Survey: 
Item Related to Student Cherectvristics 



Ciny u s 



4> 

I 



ERIC 





North 


South 


Wolf son 


Medical Center 


NONieetead 


Coltegi 


••Wide 




Nurtier 


Parcent 


NuRber 


Percent 


Nu*er 


Percent 


Niflfaer 


Percent 


MtMber 


Percent 




Percent 








27. 


How Often Are Tou Late For ClaeeT 












Atnost Always 


279 


7.1 


345 


6.3 


85 


6.7 


81 


6.5 


4 


2.3 


794 


6.5 


Olt«n 


577 


9.6 


416 


7.6 


106 


B.4 


75 


6.1 


15 


8.8 


W9 


8.2 


Sonet iaes 


827 


21.2 


885 


16.1 


257 


20.3 


188 


15.2 


25 


14.6 


2,182 


18.1 


Almost Never 


?,425 


62.1 


S.S5S 


70.0 


816 


64.6 


892 


72.2 


127 


74.3 


8.113 


67.2 


lot«l 


3<908 


100.0 


5,499 


100.0 


1.264 


100.0 


1,236 


m,ti 


171 


100.0 


12,078 


100.0 


Ntssing e 6S1 






























za. 


So fur, Roi 


1* Mould feu lete Tour Perf ome 


e in Thta CleaeT 










CKcetlent 


79^ 


19.5 


972 


17.0 


278 


21.0 


226 


17.8 


43 


24.3 


2,315 


18.4 


Good 


2.089 


51.S 


2.839 


49.7 


690 


52.3 


694 


54.7 


85 


48.0 


6,397 


51.0 


f»ir 


968 


2S.8 


1,502 


26.3 


286 


21.7 


273 


21.5 


38 


21.5 


3.067 


24.4 


Poor 


172 


4.2 


338 


5.9 


53 


4.0 


70 


5.5 


8 


4.5 


641 


5.1 


Don't KnoM 


51 


1.2 


61 


1.1 


13 


1.0 


6 


0.5 


S 


1.7 


134 


•1 


Total 


4,C74 


100.0 


5,712 


100.0 


1.320 


100.0 


1,269 


100.0 


177 


100.0 


12,552 


100.0 


Missing = 177 






























29. NOM 


Difficult is This Coutm I 




1 To Other C 


oureee Tou Have Takenr 










More Difficult 


924 


22.7 


1,743 


30.6 


404 


30.8 


512 


40.1 


56 


31.6 


3.639 


29.0 


About the Same 


1.984 


48.8 


2.852 


50.0 


715 


54.4 


591 


46.2 


80 


45.2 


6,222 


49.7 


Less Difficult 


1.156 


28.5 


1.103 


19.4 


195 


14.8 


175 


13.7 


41 


23.2 


2.670 


21.5 


lot si 


4.064 


100.0 


5,698 


100.0 


1.3U 


100.0 


1.278 


100.0 


177 


100.0 


12.551 


100.0 


Missins a 198 
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Table S 

CcontinMed) 



Results of Fell 1990 Pilot of Student Feedbeck Survey: 
tteae Reteted to Studsnt Oisrecteri sties 



Cenpus 



North Soutli liotfsen IMleel C«tter Komteod College- Vide 



Maiter Percent UMber Percent Nuober Percent Uigtmr Percent mmtmr Percent l^^ier Percent 







TIkt Drnount Of Mark In n»is Coutm i 




To The 


: In Other 




I Tou Km* 


TaksnT 






Greater 


936 


25.1 


1.556 


27. i 


375 


28.5 


521 


41.0 


57 


52.2 


3.445 


27.5 


About the Same 


2.292 


S6.5 


3.169 


55.7 


800 


60.8 


628 


49.4 


102 


57.6 


6,991 


55.9 


less 


828 


20.4 


969 


17.0 


140 


10.7 


122 


9.6 


18 


10.2 


2,077 


16.6 


Total 


4,0S6 


100. 0 


5,694 


100.0 


1.315 


100.0 


1,271 


100.0 


177 


100.0 


12,515 


100.0 


Missing « 216 






























It. 


Iflwt Do You Think Mmit The matt 


er Of Students In This ClassT 










Too 4any 


^kl 


13.4 


1,072 


18.8 


148 


11.2 


sss 


26.5 


13 


7.5 


2.111 


16.9 


Rif^t UiMstoer 


1,167 


83.0 


4.468 


78.5 


1.097 


85.5 




72.0 


156 


88.2 


10,005 


79.8 




US 


3.6 


168 


2.9 


72 


5.5 


22 


1.7 


8 


4.5 


418 


3.5 


Total 


4.058 


100.0 


5, roe 


100.0 


1,517 


100.0 


1,274 


100.0 


177 


100.0 


12.554 


100.0 


Nts&ing s !9S 
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. Uhy Are You Taking This Course? 












Requirement 


2.806 


69.5 


3,947 


69.6 


900 


68.4 


1.046 


85.4 


139 


78.5 


8.858 


70.9 


Elective 


S23 


12.9 


836 


14.7 


187 


14.2 


60 


4.8 


17 


9.6 


1,625 


15.0 


Upgrade SItilU 


16$ 


6.2 


188 


5.5 


85 


6.5 


9S 


7.4 


8 


4.5 


540 


4.5 


Personal Interest 


S23 


8.0 


459 


8.1 


107 


8.1 


28 


2.2 


9 


5.1 


926 


7.5 


Other 


220 


5.4 


245 


4.5 


59 


5.0 


za 


2.2 


4 


2.5 


536 


4.5 


Total 


4.040 


100.0 


5.675 


1(M>.0 


1,516 


100.0 


1.255 


100.0 


177 


100.0 


12,465 


10O.0 



Ntsstng * 266 



Tabte 3 

(continued) 

Results of fall 1990 Pilot of Student Feedbock Survey: 
IteM fteieted to Student CherecterUtice 



north South Hot fun Hedfcel Center Monetteed CoUege^ttldfe 



ikffter Percent Ihjiber Percent Ifaiiier Percent lhalwr Percent Ihiiter Percent fMtm' Percent 



XS* Kme Tou iegietered for Tbie Cautms leforeT 



Yes 


578 


U.3 


816 


14.4 


252 


19.2 


120 


9.6 


26 


14.6 


1.792 


U.4 


Mo 


5,469 


85.7 


4.670 


85.6 


1.061 


80.8 


1.157 


90.4 


152 


85.4 


10.689 


8S.6 


Total 


4,047 


100.0 


5.686 


IMt.Q 


1,513 


100.0 


1,257 


100.0 


178 


100.0 


12.481 


100.0 








— — — — 






















— - — ..... 
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1 A 
















Male 


1,694 


42.2 


2.520 


44.5 


505 


58.7 


289 


25.2 


50 


28.5 


5.056 


40.8 




2. 525 


57.8 


5.145 


55.5 


798 


61.3 


958 


76.8 


127 


71.7 


7.549 


59.2 


Iota! 


4.017 


100.0 


5.665 


100.0 


1.501 


1(N).0 


1.247 


100.0 


177 


^QO.0 


12.405 


100.0 


Hissing £ 524 


































55. Mou 


i Old Are Tour 














16 or Under 


662 


21.5 


1,147 


20.2 


163 


12.4 


47 


5.7 


41 


25.0 


2.260 


18.1 


19 - 24 


2,308 


57.0 


5.528 


62.1 


710 


54.0 


518 


41.5 


85 


47.8 


7.149 


57.5 


?S 51 


521 


12.9 


604 


10.6 


228 


17.4 


369 


29.4 


20 


11.2 


1.742 


14.0 


12 40 


241 


5.9 


269 


4.7 


142 


10.8 


254 


18.6 


19 


10.7 


905 


7.2 


41 or Over 


118 


2.9 


157 


2.4 


71 


5.4 


88 


7.0 


IS 


7.1 


427 


5,4 


!ol»l 


4.050 


100.0 


5.685 


100.0 


1.514 


100.0 


1.256 


100.0 


178 


100.0 


12.485 


100.0 



Missing = 246 
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Table S 

Ccontinued) 



Results of Fall 1990 Pi tot of Student feedbvk Survey; 
ItecBs Related to Student CharMteristica 



Catqpua 







North 


South 


Wolf son 


Nedicel 


Center 


Nonestead 


Coilegi 


l-Wfd> 






NiAter 


Percent 


NiAber Percent 


NtMber 


Percent 


Ntater 


Percent 


KuiAier 


Percent 


Miabar 


Percent 










36. Pic 


Me indicate lew Tou Identify Tourself 












American 


Indian 


86 


2.1 


120 


2.1 


35 


2.7 


14 


1.1 


6 


3.4 


261 


2.1 


Asian 




231 


5.7 


240 


4.2 


49 


3.8 


62 


5.0 


6 


3.4 


588 


4.8 


Stack 






53.2 


394 


7.0 


174 


13.3 


372 


29.8 


22 


12.4 


2.296 


18.5 


White 




1.49S 


37.2 


3.744 


66.4 


721 


55.3 


597 


47.8 


114 


64.4 


6.671 


53.8 


Other 




8^9 


21.8 


1.143 


20.5 


324 


24.9 


204 


16.3 


29 


16.4 


2.579 


20.8 


Total 




4.025 


100.0 


5.641 


100.0 


1.305 


100.0 


1.249 


IM.O 


177 


100.0 


12.595 


100. 0 


Missing 




































37. 


Is Tour Ethnic tteritaoe Miepantc7 












Yes 






45.7 


5.561 


60.2 


912 


70.8 


439 


35.2 


45 


25.4 


6,534 


52.9 


No 




2,262 


56.5 


2,238 


59.8 


376 


29.2 


807 


64.8 


132 


74.6 


5.815 


47.1 


T0t8t 




4.019 


100.0 


5,619 


100.0 


1.288 


100.0 


1,246 


100.0 


177 


100.0 


12.M9 


100.0 


Hissing 


= 580 






























36. 


Po You Nm« Fwily C 


emitae 


nts That Interfere With i 


OM Well 


Vou Oo In Cte 










Tes 




i.in 


29.8 


1.569 


28.4 


442 


34.6 


558 


45.2 


60 


35.5 


5.802 


31.3 


No 




2,r65 


70.2 


5,965 


71.6 


837 


65.4 


67B 


54.8 


110 


64.7 


8,555 


68.7 


Totat 




5.958 


100.0 


5.534 


100.0 


1.279 


100.0 


1,236 


KW.O 


170 


100.0 


12.157 


100.0 
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fable 3 

(continued) 



Results of Fall 1990 Pilot of Student feedback Swvey: 
ltea» eelated to Student Ctiaracteristtcs 





li»rth 


south 


Wot f son 


Nedicat 


Canter 


NcaK 


stead 


Cetlega-Wfda 






Parccnt 


Niater 


Parcent 




Percent 


Vurtier 


Percent 


MUHlMr 


Percent 


Voter 


Percent 






19. 


NoH Many 


ioura Par 1 


M( Be Tou UKUitty Uarfc At Tour Mb! 










No Job 


i.oor 


25.0 


1,379 


24.3 


393 


30.0 


350 


27.7 


46 


26.0 


3.175 


25.5 


1 - 20 Hours 


914 


22.6 


1.409 


24.8 


263 


20.0 


511 


24.6 


45 


25.4 


2,942 


23.6 


21 • SO Hours 


921 


22.S 


1.443 


25,4 


214 


16.3 


219 


17.4 


45 


25.4 


2.842 


22.8 


St ■ 40 Hours 


795 


19.7 


948 


16.7 


262 


20.0 


276 


21.9 


30 


17.0 


2.311 


18.5 


More Than 40 Hours 


400 


9.9 


1,96 


8.8 


180 


13.7 


106 


8.4 


11 


6.2 


1.193 


9.6 


lotal 


4.037 


100.0 


5,675 


100.0 


1,312 


100.0 


1.262 


100.0 


177 


100.0 


12,465 


100.0 


Hissing s 266 

. — . 
































40. NoM 


Many Credits Are Tou Trtina Tbia ScavatarT 












11 or Lms Credits 


1,407 


34.9 


2.200 


38.8 


459 


35.1 


459 


36.3 


92 


S2.0 


4.617 


37.1 


12 or More Credits 


2,628 


65.1 


J.467 


61.2 


850 


64.9 


804 


63.7 


85 


48.0 


7,834 


62.9 


Tot*( 


4.035 


100.0 


5.667 


100.0 


1,309 


100.0 


1,263 


100.0 


177 


100.0 


12.451 


100.0 


Ntssing > 278 






























41. 


Nave You Taken A Coura* IHth This fnatnictor lefore? 










Tea 


462 


11.4 


506 


8.9 


249 


18.9 


214 


17.0 




S.4 


1.437 


11.5 


Mo 


5.556 


88.0 


5,145 


90.6 


1,052 


79.9 


1,041 


82.9 


171 


96,1 


10.965 


87.9 


Don't Remeober 


24 


0.6 


28 


0.5 


15 


1.2 


1 


0.1 


1 


0.5 


69 


0.6 


Total 


4.042 


100.0 


5.679 


100.0 


1.316 




1.256 


100.0 


178 


TOO.O 


12.471 


100.0 


Missing « 258 



























I 
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Tabl«4 



Factor Results <^ Fall Tenn Student Feedback Surrey 
Based on Weightings of .30 (Mt JUsove 



FlTlr^- im the IhiftrfAiel 

.78 Concerned with my progress Gtem 5) 

.60 Znfonns regularly aJbout progress (Item 19) 

.39 Shows how material can benefit outside class (Item 6) 

.34 Available for individual help (Itero 8) 

P«eter2: Comnfnfm hi QMWQQm 

.70 butructor shows interest in subject (Item 2) 

.58 Instructor is prepared for class Qtom 1) 

.50 Demonstrates knowledge of subject (Item 15) 

FefltarS! ApmchtoMrteriri 

.75 Creates atmosphere encouraging learning (Item 14) 

.74 Makes course interesting (Item 7) 

.45 Presents material cleariy (Item 13) 

.32 Uses variety of teaching methods (Item 16) 

F«gtar4! Grertlmf PoHiTV 

.78 Discussed grading at beginning (Item 22) 

.53 Grading system was dear (Item 12)* 

FeetorB: liafffring Iff f ty tftmtT 

.68 Pays attention to comments (Item 20) 

.60 Treats me with respect (Item 21) 

.32 Encourages questions in class (Item 9) 

FaetorB: Qarftff on Cquim Obiecthwt 

.76 Objectives and what is taught agree Gtem 4) 
.55 Distributed course objectives (Item 3) 

Fftfftfflr?; FttnffUftfRMmt 

.78 Exams graded fairly QXam 1 1) 

.36 Evaluation related to material (Item 10) 

.31 Grading system clear (Item 12)* 

factor ft J^et^WB Leairina 

.52 Assignments help leaming (Item 17) 
.49 Encourages thinking for self (Item 18) 

I . 

•This item is included in more than one factor. 

Note: Item 23 (Starts class on time) did not load above .30 on any factor. The greatest 
weight was .24 on factor 2. 

-39- 
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Tablo 5 

Inter-Factor Correlations and Factor Weights 



1 2 3 4 5 


6 


7 


8 


1 - 47 65 44 61 


50 


49 


54 


2 - 66 46 58 


66 


56 


49 


3 - 46 64 


60 


59 


64 


4 - 51 


51 


58 


47 


5 


52 


58 


63 


6 




61 


54 


7 
8 






55 


Unique Variance Attributed to Each Factor 


Weighted 1.86 1.14 1.72 1.52 1.09 
Unweighted .71 .54 .55 .58 .44 
Percent of 

Weighted Total 5.9 3.4 5.4 4.8 3.4 


1.18 
.43 

3.7 


1.10 
.40 

3.5 


.65 
.31 

2.0 


Total Variance Attributed to Each 


Factor 






Weighted 15.62 15.18 18.79 11.42 16.01 
Unweighted 6.22 6.28 7.38 4.62 6.48 
Pgccoh^ of 

Weighted Total 49.1 47.7 59.1 35.9 50-3 


14.79 
5.94 

46.5 


14.21 
5.68 

44.7 


13.63 
5.63 

42.8 


Total communality : Weighted = 31.82 / Unweighted 


= 12.66 
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Inter- 1 tM CorrcUtton Natrtx 
Items 1 23 
Fall 1990. Student feedback Diita 

M« 11,625 





\ 2 


1 


4 


5 


6 


7 


8 


9 


10 


11 


12 


15 


14 


15 


16 


17 


18 


19 


20 


21 


22 


23 


1 


55 


4S 


48 


57 


sr 


46 


35 


57 


41 


37 


37 


50 


45 


47 


34 


35 


35 


27 


39 


39 


34 


37 


2 




4} 


44 


40 


41 


49 


57 


40 


41 


36 


54 


46 


47 


50 


35 


31 


36 


26 


40 


39 


30 


26 


5 






S6 


5a 


y> 


40 


55 


55 


41 


58 


59 


46 


42 


40 


32 


K 


35 


29 


37 


33 


36 


26 










45 


41 


47 


40 


59 


50 


44 


41 


55 


48 


44 


36 


42 


40 


55 


42 


39 


37 


28 


b 










S5 


54 


bl 


46 


IT 


40 


40 


49 


55 


54 


40 


45 


49 


58 


52 


46 


32 


25 


6 












57 


42 


44 


56 


56 


57 


^8 


55 


17 


41 


41 


46 


44 


47 


39 


30 


22 


/ 














47 


49 


42 


41 


41 


61 


71 


44 


48 


45 


48 


44 


50 


45 


33 


28 


a 
















46 


59 


If 


58 


44 


46 


55 


35 


41 


45 


42 


49 


43 


32 


29 


V 


















45 


58 


57 


48 


52 


5V 


58 


41 


46 


58 


54 


46 


51 


25 


111 




















5? 


4^ 


SO 


45 


45 


5? 


41 


5V 


?9 


41 


40 


56 


28 


H 






















54 


U 


44 


4U 


54 


58 


5/ 


M 


44 


42 


58 


28 


1.' 
























40 


4S 


38 


55 


58 


5/ 


58 


42 


40 


58 


26 


15 


























65 


47 


45 


48 


4/ 


42 


51 


46 


59 


35 


U 




























47 


49 


49 


52 


45 


S5 


49 


55 


29 


lb 






























17 


16 


H 1 


CO 


H 1 


Hv 


li; 


tn 

JV 


16 
































39 


4U 


58 


39 


35 


29 


2« 




































52 


45 


44 


59 


54 


25 


16 




































45 


51 


42 


32 


2« 


19 






































49 


57 


32 


24 


^0 








































59 


37 


30 


?1 










































40 


SS 


?^ 












































33 


24 














































Catnmmali 1 1)^ 


.SI .s; 


.SO 


.69 


.?1 


.49 


.70 


.43 


.46 


.49 


.67 


.61 


.61 


.73 


.48 


.36 


.51 


.54 


.52 


.67 


.54 


.64 


.23 


Weight 


^05 


1.W 


5.20 


5.40 


1.9S 


5.57 


1.75 


1.85 


1,95 


5.01 


2.57 


2.55 


5.73 


1.92 


1.57 


2.06 


2.19 


2.09 


5.04 


2.19 


2.78 


1.30 



AIUH.Z.S 



ERIC 



r,r> 



AMlyvlt of varianc* and Group Maana for Item 28 1 
Farfomanea in claaa n ■ 12,491 



Group It Bxcallant Parformanca N « 2,305 

Group 2t Good Parfomance n - 6,376 

Group 3t Pair parfonnanco M - 3,043 

Group 4t Poor Parfonnanco N • 635 

Group 5t Pen't Know Parformanca N • 132 





Dapandant Variable 


F-Ratio 


R* 


r 


p <.01 


1: 


Focua on individual 


324.86 


.094 


.31 




2x 


CMq^tanca in Claaaroom 


87.39 


.027 


.16 


« 


3s 


Approach to Material 


264.26 


.078 


.28 


* 


4t 


Grading Policy 


114.98 


.036 


.19 


« 


5j 


Listening to Students 


168.39 


.051 


.23 


• 


6t 


Clarity on course Objectives 


145.96 


.045 


.21 


« 


7: 


Fairneaa of Exams 


203.52 


.061 


.25 


* 


8: 


Active Learning 


200.75 


.060 


.24 


• 



Means by croup and Factor 




Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Group 


1 


2 


3 


4 


5 


6 


7 


8 


1 : Excellent 


0.42 


0.21 


0.36 


0.26 


0.32 


0.29 


0.35 


0,34 


2 : Good 


0.09 


0.04 


0.09 


0.04 


O.OS 


0.05 


0.06 


0.07 


3 1 Fair 


-0.34 


-0.17 


-0.28 


-0.18 


-0.24 


-0.22 


-0.25 


-0.26 


4 : Poor 


-0.70 


-0.38 


-0.74 


-0.40 


-0.50 


-0.47 


-0.53 


-0.58 


5: Don't Know 


-0.62 


-0.58 


-0.65 


-0.73 


-0.61 


-0.63 


-0.76 


-0.53 



Statistically Significant Mean Differences 



1-2 


1-2 


1-2 


All 


1-2 


1-2 


1-2 


1-2 


1-3 


1-3 


1-3 




1-3 


1-3 


1-3 


1-3 


1-4 


1-4 


1-4 




1-4 


1-4 


1-4 


1-4 


1-5 


1-5 


1-5 




1-5 


1-5 


1-5 


1-5 


2-3 


2-3 


2-3 




2-3 


2-3 


2-3 


2-3 


2-4 


2-4 


2-4 




2-4 


2-4 


2-4 


2-4 


2-5 


2-5 


2-5 




2-5 


2-5 


2-5 


2-5 


3-4 


3-4 


3-4 




3-4 


3-4 


3-4 


3-4 


3-5 


3-5 


3-5 




3-5 


3-5 


3-5 


3-5 



Educationally significant Mean Differences 

1-3 1-4 1-3 1-4 1-3 1-3 1-3 1-3 

1-4 1-5 1-4 1-5 1-4 1-4 1-4 1-4 

1- 5 2-5 1-5 2-5 1-5 1-5 1-5 1-5 

2- 4 2-4 3-5 2-4 2-4 2-4 2-4 
2-5 2-5 2-5 2-5 2-5 2-5 

3-5 3-5 
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TabU 8 



Analysis of variance and Group Means for Item 29: 
Difficulty of Courss compmrmd to others M • I2(.468 



Group 1 1 
Group 2: 
Group 2% 



More Difficult 
About ths same 
Less Difficult 



N - 3,613 
N - 6,205 
N » 2,650 



Dependent Variable 



1: Focus on Individual 

2: competence in classroom 

3; Approach to Material 

4; Grading Policy 

5t Listening to students 

6t Clarity on course objectives 

7: Fairness of Exams 

9t Active Learning 



F-Ratio 




r 


95.79 


.015 


.12 


11.75 


.002 


.04 


193,76 


.030 


.17 


35.56 


.006 


.08 


77.22 


.012 


.11 


48.26 


.008 


.09 


118.23 


.019 


.14 


38.11 


.006 


.08 



p <.01 



Group 








Means by Group and Factor 












Factor 

1 


Factor 
2 


Factor Factor Factor 
3 4 5 


Factor 
6 


Factor 
7 


Factor 
8 


It More 
2: sane 
3 : Less 


Oiff . 
Oiff . 


-0.19 
0.06 
0.10 


-0.07 
0.03 
0.00 


-0.27 -0.12 -0.18 
0.09 0.05 0.05 
0.15 0.02 0.10 


-0.14 
0.06 

0.05 


-0.21 
0.08 
0.10 


-0.12 
0.06 
0.04 


Statistically Significant Mean Differences 






1-2 
1-3 


1-2 
1-3 


All 1-2 1-2 
1-3 1-3 


1-2 
1-3 


1-2 
1-3 


1-2 
1-3 


Educationally significant Mean Differences 






none 


none 


none none none 


none 


none 


none 
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Table 9 



AnaXyals of Variance and croup Means for item 30 < 
anount of work compared to other Courses M « 12^454 



<3roup It More work in This course N « 3^428 
Group 2 1 About the same Amount n - 6|963 
Group 3s Less Work in This course N » 2^063 





Dependent variable 


F-Ratio 


R» 


r 


P 


<.01 


Is 


Focus on individual 


47.01 


.007 


.08 




» 


2: 


competence in Classroom 


20.33 


.003 


.05 




* 


3: 


Approach to Material 


30.18 


.005 


.07 




* 


4: 


Grading Policy 


29.25 


.005 


.07 




* 


5t 


Listening to students 


18.01 


.003 


.05 




* 


St 


clarity on course objectives 


31.58 


.005 


.07 




• 


7t 


Fairness of Exams 


40.32 


.006 


.08 




• 


Bt 


Active Learning 


95.11 


.015 


.12 




* 


Means by Group and Factor 




Factor Factor Factor 


Factor 


Factor Factor 


Factor 


Factor 


Group 


12 3 


4 


5 6 




7 


8 


1 : More 


work 0.00 -0.01 -0.07 


-0.04 


-0.05 -o.oe 




-O.ll 


0.07 


2: Same 


0.05 0.03 0.06 


0.05 


0.04 0.06 




0.07 


0.06 


3: Leas 


work -0.19 -0.13 -0.09 


-0.12 


-0.09 -0.11 




-0.04 


-0.27 


Statistically significant Mean Differences 




All 3-1 2-3 


All 


2-3 2-3 




All 


3-2 




3-2 2-1 




2-1 2-1 






3-1 


Educationally Significant Mean Differences 




none none none 


none 


none none 




none 


none 
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Tftblo 10 

Anslyii* of Vafianes and Group Moana for lt«m 31 i 
Mioibar of students in Claaa N « 12,475 





Oroup It Too 


Many N 


- 2,096 






Group 2t Ri9ht Number H 


* 9,965 






Group 3: Too 


Few N 


- 414 






Dependent Variable 


F-Ratio 


R' 


r 


It 


Focua on individual 


29.27 


.005 


.07 


2t 


Competence in classroom 


42.52 


.007 


.08 


3t 


Approach to Material 


41.99 


.007 


.08 


4t 


Grading Policy 


37.20 


.006 


.08 


5i 


Listening to students 


48.10 


.008 


.09 


6: 


clarity on course objectives 


36.85 


.006 


.08 


7i 


Faimeat of Exams 


41.33 


.007 


.08 


9t 


Active Learning 


16.63 


.003 


.05 



p <.0l 



Means by croup and Factor 


Group 


Factor 
1 


Factor Factor Factor Factor 
2 3 4 5 


Factor 
6 


Factor 
7 


Factor 
8 


1: Too many 
2t Right No. 
3t Too few 


-0.12 
0.03 
-0.23 


-0.03 -0.10 -0.05 -O.IS 
0.01 0.03 0.02 0.04 
-0.44 -0.35 -0.39 -0.29 


-0.05 
0.02 
-0.39 


-0.07 
0.03 
-0.40 


-0.09 
0.03 
-0.13 


Statistically Significi^nt Mean Differences 




1- 2 

2- 3 


1- 3 All All All 

2- 3 


All 


All 


1- 2 

2- 3 


Educationally significant Mean Differences 




none 


none none none none 


none 


none 


none 
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T«bl» 11 



Analysis of V«ri«nc« and Group M«ao« for Sizm at Court* 

<at End of S«8»atar) 
H ■ €40 ssetiona 



Sizs It 

size 2t 
Size 3{ 
Size 4t 
Size 5t 



10 or Student » » • 3* 

11-20 student* N - 200 

21-30 Student* » • 229 

31-40 Students M • 114 

More Than 40 Students H • $3 



Dependent Variable 



1: FOCUS on Individual 

2: competence in Classroom 

3t Approach to Material 

4t Grading Folicy 

5t Listening to students 

6t clarity on Course objectives 

7: Fairness of Exams 

9 1 Active Learning 



F-Ratio 


R> 


r 


p <.01 


7.04 


.042 


.20 


• 


1.00 


.006 


.07 




2.96 


.018 


.13 




0.94 


.006 


.07 




5.53 


.034 


.18 


• 


2.26 


.014 


.12 




1.07 


,007 


.08 




10.74 


.063 


.25 


* 











Means by Group and 


Factor 








Size 




Factor 

1 


Factor 
2 


Factor 
3 


Factor 
4 


Factor 
5 


Factor 
6 


Factor 

7 


Factor 
8 


1: 10 
2t 11 
3: 21 
4t 31 
5t 40 


or less 

- 20 

- 30 

- 40 
♦ 


0.51 
0.15 
0.01 
-0.33 
-0.14 


-0.08 
0.00 
0.01 

-0.09 
0.21 


0.26 
0.12 
-0.03 
-0.24 
0.01 


-0.16 
-0.04 
-0.01 
0.02 
0.20 


0.52 
0.11 
0.01 
-0.27 
-0.15 


0.06 
0.04 
0.04 
-0.25 
0.12 


-0.12 
0.00 
0.00 

-0.08 
0.22 


0.50 
0.25 
0.02 
-0.38 
-0.23 





Statistically significant Mean 


Differences 






4-3 


none none none 


1-4 none 


none 


4-3 


4-2 




1-5 




4-2 


4-1 




1-3 




4-1 


5-1 




2-4 




5-2 


3-1 








5-1 










3-2 










3-1 




Educationally Significant Mean 


Differences 






none 


none nona none 


none none 


none 


none 
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Tabl« 12 



Aiimly«ift of variance and Oroup itaans for Item 32 s 
Raaaona for Taking couraa M - 12,403 



Group Is 
Group 2s 
Group 3s 
Group 4s 
Group 5t 



Roquiroman^ for Dagrea N - 8,794 
Elactiva for Dagraa N ^ 1,618 
Upgrada Job Skills N « 537 
Parsonal Intaradt N • 921 

other Raaaona N ■ 533 



Dependant variable 



F-Ratio 



R» 



p <.01 



It 


Focus on individuaL 


22.46 


.007 


.08 


* 


2t 


coiap«t«nc« in classroom 


8.24 


.003 


.05 


* 


3s 


Approach to Hatari&l 


17.60 


.006 


.08 


• 


4t 


Grading Policy 


4.97 


.002 


.04 


* 


St 


Lis toning co studants 


6.74 


.002 


.04 


* 


6t 


clarity on course objectives 


4.73 


.002 


.04 


* 


7x 


Fairness of Exams 


5.86 


.002 


.04 


• 


8t 


Active Learning 


5.70 


.002 


.04 


« 




Means by Group and 


Factor 








Factor Factor Factor 


Factor 


Factor 


Factor Factor 


Factor 


Group 


12 3 


4 


5 


6 7 


8 



It Requirement -0.05 >0.01 ->0.04 0.01 -0.02 -0.01 0.00 -0.02 

2: Elective 0.03 -0.04 0.04 -0.06 -0.02 0.01 -0.01 0.01 

3: Upgrade 0.22 -0.08 0.04 0.00 0.04 -0.11 -0.11 0.11 

4: Personal 0.21 0.13 0.24 0.00 0.15 0.09 0.11 0.12 

5: Other 0.03 -0.16 -0.07 -0.15 -0.06 -0.10 -O.ll -O.Ol 



Statistically significant Mean Differences 



1-2 
1-3 

1- 4 
5-4 
5-3 

2- 4 
2-3 



1- 4 

2- 4 

3- 4 

4- 5 
1-5 



1-2 

1- 4 

2- 4 

3- 4 
5-4 



1-2 
1-5 
4-5 



4-1 
4-2 
4-5 



3-4 
5-4 



4-1 
4-2 
4-3 
4-5 



1-3 
1-4 



Educationally Significant 


Mean Differencea 






none none none none 


none none 


none 


none 
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Tabltt 13 

Analyaia of Varianc* for Tims of court* 
H » 627 Sections 



N - 164 
N « 239 
N • 77 
N « 73 
N - 74 



Tim li early Nornin? (7t00 - 8t00 a.D.) 

TiSM 2t llomia9 (8t01 - lltOO a.m.) 

Tlse 3t Lunch hour 1 1^*01 a.m. - 12:59 p.jn*> 

Tina 4t Aftamoon ( ItOO - 4tS9 p.m.) 

Tine 5t Evanina (StOO p.m. or latar) 





Dopandant Variabla 


F-Ratio 


R» 


r 


It 


Focus on Individual 


0.11 


.000 


.00 


2t 


cmnpatenca in claasroom 


O.ll 


.001 


.03 


3t 


Approach to Natarial 


0.41 


.003 


.05 


4t 


Grading Policy 


1.44 


.009 


.09 


Si 


Listening to students 


0.99 


.006 


.08 


6t 


clarity on course Objectives 


0.31 


.002 


.04 


7: 


Fairness of Exams 


0.19 


.001 


.03 


8t 


Active Learning 


0.30 


.002 


.04 



Means by Group and Factor 





Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Factor 


Tine 


1 


2 


3 


4 


S 


6 


7 


8 


Early 


0.00 


-0.01 


0.03 


-0.09 


0.05 


0.02 


-0.03 


-0.04 


Morning 


0.03 


-0.01 


O.Ol 


0.03 


-0.03 


-0.01 


0.00 


0.02 


Lunch 


-0.03 


-0.02 


-0.01 


-0.14 


0.01 


0.06 


0.03 


0.03 


Afternoon 


-0.04 


0.04 


-0.12 


0.04 


-0.12 


0.08 


0.04 


0.08 


Evening 


-0.03 


0.06 


0.07 


0.19 


0.17 


-0.07 


0.08 


-0-06 
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Tabl* 14 



Analysis ot vsriancs and aroup Ksans for instructor Rank 

M « 627 Ssetlons 







Rank 1 


Professor 


N 


m 


264 










Rank 2 




fir N 




117 










Rank 3 


aftfiis^An^ PirofossoF 


If 




79 










Rank 4 


T nil time tor 


Kw 




64 










Rank 5 






m 


102 








Dependent Variable 


F'Ratio 






r 


p <.01 




It 


Focus on 


Individual 


14.56 


.086 




.29 


* 




2t 


competence in classroom 


1.29 


.008 




.09 






3: 


Approach to Material 


5.47 


.034 




.18 


* 




4t 


Grading Policy 


2.79 


.018 




.13 






St 


Listenxng 


to students 


6.45 


.040 




.20 


* 




6: 


clarity on course objectives 1.34 


.012 




.11 






7s 


Fairness 


of Exams 


2.26 


.014 




.12 






Si 


Active Learning 


7.28 


.045 




.21 


• 




Means by Group and Factor 






Factor Factor 


ractor Factor 


Factor 




Factor 


Factor 


Factor 


Group 


1 2 


3 4 


5 




6 


7 


8 


It 


Professor 


-0.27 -0.02 


-0.16 -0.06 


-0.15 




-0.05 


0.00 


-0.16 


2: 


Asso. Sr« 


-0.02 -0.04 


-0.01 0.01 


-0.07 




-0.03 


-0.03 


-0.02 


3: 


Assistant 


0.50 0.22 


0.32 0.21 


0.37 




0.19 


0.21 


0.43 


4t 


Instructor 


0.47 0.08 


0.31 0.23 


0.34 




0.22 


0.17 


0.32 


5: 


Associate 


0.00 -0.07 


-0.03 -0.19 


0.00 




-0.09 


-0.19 


-0.07 


Statistically significant Mean nifferencee 






4-1 none 


1-4 none 


1-4 




none 


none 


3-2 






4-2 


1-3 


1-3 








3-5 






4-5 




2-3 








3-1 






3-1 












4-1 






3^2 


















3-5 














Educationally Significant Mean Differences 






4-1 none 


none none 


1-3 




none 


none 


3-5 






3-1 












3-1 



3-2 
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Analyiis of VAriane* for subject N«tt«r 
N ■ 309 sections 





APB Applied Biology 




N 


• 21 






CBM ciMmistry 






K 


- 20 






SMC English 






N 


- 94 






SHS English as < 


a Second Language 


N 


m 33 






HUH RiUBAnitiafl 






N 


m 24 






MAC ttet hraafric a • 

#bSW^^ Cv^K ^Ib^BW***** W<A W IP 


-calculus & 


Prfi~C&lculus 

A an ^ir )!■> M> WPk 


N 


- 32 






MAT MA^homA^lca 






N 


» 30 






NUR Nursing 






N 


- 35 






PSY Fsvcholoov 






N 


- 20 






Dependent variable 


F-Rstio 






r 


p <.01 


1: 


Focus on Individual 


9.06 


.19 




.44 




2: 


competence in Claseroom 


0.50 


.01 




.10 




3: 


Approach to Material 


1.89 


.05 




.22 




4: 


Grading Policy 


2.37 


.06 




.25 




5s 


Listening to students 


1.93 


.05 




.22 




6t 


Clarity on course Objectives 


1.23 


.03 




.17 




It 


Fairness of Exams 


3.30 


.08 




.28 




8s 


Active Learning 


6.92 


.16 




.40 


• 



Means by Group and Factor 



Subject 


Factor 
1 


Factor 
2 


Factor 
3 


Factor 
4 


Factor 
5 


Factor 
6 


Factor 

7 


Factor 
8 


APB 


-0 


.52 


-0, 


.10 


-0 


.52 


-0. 


07 


-0.43 


-0. 


37 


-0. 


36 


-0. 


91 


CHN 


-0 


.23 


0. 


.23 


-0 


.12 


0. 


37 


-0.30 


0. 


41 


0. 


33 


0. 


04 


ENC 


0 


.19 


-0, 


.50 


0 


.05 


-0. 


31 


0.00 


0. 


02 


-0. 


28 


0. 


10 


EMS 


0 


.69 


0, 


.00 


0 


.32 


0. 


24 


0.40 


-0. 


02 


0. 


13 


0. 


48 


HUN 


-0 


.52 


-0, 


.05 


0 


.02 


-0. 


10 


-0.22 


-0. 


18 


0. 


08 


-0. 


68 


KAC 


-0 


.67 


-0. 


.16 


-0 


.31 


0. 


09 


-0.07 


-0. 


06 


0. 


24 


0. 


11 


HAT 


-0 


.25 


0. 


.01 


-0 


.06 


0. 


23 


0.06 


0. 


13 


0. 


43 


0. 


33 


NUR 


0 


.57 


0. 


.02 


0 


.24 


0. 


08 


0.26 


-0. 


10 


-0. 


19 


0. 


17 


PSY 


-0 


.20 


0. 


.29 


0 


.05 


0. 


34 


-0.11 


0. 


30 


0. 


40 


-0. 


55 



Statistically significant Mean Differences 



none none none none none ENC-MAT APB-chm 

APB-ENC 
APB-HAC 
APB-NUR 
APB-HAT 
APB-EHS 
HUM-EKC 
HUM-KAC 
HUM-NUR 
HUM-MAT 
HUM-EMS 
PSY-MAT 
PSY-EMS 



Educationa^lly Significant Mean. Differences 



same none none none none none sane saxne 



EMS-MAC 
EMS-APB 
ENS-HUM 
EMS-MAT 
ENS-CBM 
EMS-PSY 
NUB-MAC 
NUR-APB 
NUR-HUM 
NUR-MAT 
EMC-MAC 
EMC -APB 
ENC -HUM 
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Table 16 



Results of felt 1990 Pilot of Student feedbeck Surveys 
I tens fteleted to fleectione to Survey 



Ce^Fius 



iorth South Ifolfeon Hedieel Center HcMeteed CoUege-Wtde 



MMber ^rcent Niflter Percent iieiter Percent Mtuber percmt Nuiter Percent mmtr Percent 











42. 


This Quntior 


mire Is 














foo Long 


T,1J6 


2t.t 


1.767 


31.2 


461 


35.2 


477 


38.0 


62 


34.8 


3,905 


31.4 


AtMut Risht 


2.798 


69.4 


3,766 


66.5 


819 


62.6 


754 


60.1 


H5 


64.6 


8,252 


66.3 


Too Short 


100 


2.S 


13S 


2.9 


29 


2.2 


24 


1.9 


1 


0.6 


287 


2.3 


Total 


4.034 


100.0 


S,666 


100.0 


1,309 


100.0 


1.255 


100.0 


178 


100.0 


12.442 


100.0 


Missins ' 2S7 


































43. 


This Quest iomir* Wa* 














Easy 


3,«6 


97.7 


S,420 


98.3 


1,215 


96.3 


1.186 


97.9 


170 


98.8 


11.817 


97.9 


Hard 




2.3 


96 


1.7 


47 


3.7 


26 


2.1 


2 


1.2 


2ftO 


2.1 


Total 


3.915 


100.0 


5.516 


1(».0 


1,262 


100.0 


1,212 


100.0 


172 


100.0 


12,077 


100.0 



fitsstng s 652 
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Easy to understand - OJcay the way it is 
Too long/too many questions 

Asks too many personal questions about respondent 

Race identification of respondent is unnecessary 

Add more questions concerning instructor's performance 

Add questions regarding course content and purpose 

Should ask questions about instruction materials 

Include questions regarding instructor's evaluation of students 

Add questions regarding students' behavior 

Include questions about facilities 

Wording should be so that it applies to any situation 
(not all questions pertain to all class situations) 

Need more "in-between" choices for answers 

Add a choice "not applicable" for questions 1-23 

First 23 questions need to be amended to give a more 
fair rating of instructors 

Questions need to be more direct and concise 
Difficult to understand 
Need to improve answer sheet 
Wording is too formal. 

Add "Would you take another class w/this instruct 
Question 10 was difficult to understand 
Where is #44? 

Questions at end are out of sequence 
Too redundant 

Instructor could easily identify student by the 
handwriting on the last page 

Students felt the questionnaire was a waste of their time 



269 

99 

61 

24 

25 

11 
7 
1 
2 
1 

18 

12 
7 

1 
13 
11 
2 
1 
2 
1 
18 
2 
11 

3 
9 
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student Feedback Questionnaire 



Appendix A 

(I of 2) 



icvi 
TUL. NO 



s fjntt yw Che 
wffl be givca o die 



to 

if ANONYBfOUS 
iasmiaoT \sa^ AFTER 



wdt be isepi CONnOEN- 
gDufe hn been mbinlBBBd. 



SteS^owMrapTO » eadi i» by 
{Rvv^ted. 

Piesse bitfaiile in the 3 dtefe tocaiton code under xitciamft0eaHam Mmmter oAmua A, B. C and the 5 
d^c»ne sequence tnimber under columns D, E, R G. H. Put a aw (O under cotunm I and a one (I) 
under j. 



A - 



B - 



Please use ite ftiltowisg scale to respond to 
items 1 m 2S 

Strongly agree Hba strongly agree «nih the 

as it applies to tbis 

Imuuooe 

Agree «m agice more than you 

dingrce wttb the stannaa at 
it appuea to ifata instxvaoc 

Disagree \bu disagree more than you 

agree wtth dK susmem at it 
applffff B> this imtrucBOc 

Strongly disagree \bu sBongiy dba^ w4di die 
^ ^ii^yi w #m as it applies to this 
instrvdoc 



I. 
2. 

3. 

4. 

5. 

6. 



8. 
9. 
10. 

It. 



The insnuaor is tucyai e d for daw. 

The insnuaor shows interest in tbc subfea. 

The insmiCGor dlstxibuicd the coutsc objectives/ 
comperendes. 

There is agreemem between tbe obfectivea/ com- 
petencies of this coune and what is taught. 

The instructor is ctmcexned with ray progress. 

The instiuaor shows me how the couxse material 
can benefit me beyond the daswnoom. 

The iostfuoor makes dils couisc imerBstlng. 

The iostruoor is available for individual help. 

The instrucxof encottnges questions in class. 

The opibnitons and/br odier fomis of evaluation 
are tdated to the couisc material. 

The nraminations andA>r odier fotms of evaluation 
are graded fairty. 



12. The instnicior made the giadlng system clear to me. 
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19. Tire instmcBv presents courre material clearly. 

14. The iDStrocEor cie ates a dassroom atmosphere that 
encouiagcs me to leaiu. 

15. The isstmaor demcmstraies knowtedge of the 
suofcct. 

16. The iBWMtpr uses a virtcqr of iCMhing methods 
(fbroBtmfdc; lecture, dis cussio n s , dcnonsnations. 
audiovinial aids and/or ochccs). 

17. Aasignmcon help me leant the couise material. 

18. The inscmcDorencouiagcs me CO thinltlbr myself. 

19. The iostnictpr infoimi me regularly about my 



20. The iumuctor pays attention to my comments. 

21. The inmuoor ncats nw with respca. 

Z2. The instnictor discussed the giadlng system at the 
>^""<"S of the semestec 

25. Tbe Ittstfisctor Marts class on time. 

Please use the following scale to respond to 
iteais 24 to 27 

A • Alwsjv or almost always 

B • Often 

O ■ Never or almost never 



24. How often do you come to class? 
29. How often are you prepared for class? 

26. How often do you pay attention in class? 

27. How often aic you lare for class? 

Continue on the back of this page 



2a. So fax. how wouid you rase your peribnoaace in 

A. EacdlCDX 
& Good 

CFair 

D. Poor 

E. Don I know 

29. How difficuli i» this coune compared to other 
courses you have taken? 

A. More difficult 

B. About the same 

C. Las difficult 

30 How docs the amount of work in this course com- 
pare to the amount in other courses you hav*: 
taken? 

A. The amount of work is greater 

8. The amount of work is about the same 

C. The amount of work is less 

^l. What do you think about the number of students 
in this class? 

A. V>o many studcrua 

B. The fight number of students 

C. Too few snidents 

32, VSTiv are you taking this course? 

A. AS a requiicment for a degree 

B. A* an elective for a degree 

C. To upgtade my job skills 

D. For personal interest 

E. For other reasons not listed above 

33 Have you registered for this coune bctorcr 

A. Yes 

B. No 

34 . i am a 

A .Male 
B Female 



^pendix A (2 of 2) < , , 
ntam Indlcm bam yvn idcmtfr yoondf 

A. Amertcan kKttsn or Alaskan oaiire 

B. Asian or Pidflc islander 
CBlack 

D. White 

E. Other 

37. Is your ethnic herioge Hispanic? 

A. Ves 

B. No 

58. Do you have family commiunents that imerterc 
with how well you do in class? 

A. Yes 

B. So 

39. How many hours per week do you usually work 
at your fob? 

A. i don't have a job 

B. I to 20 hours per week 

C. 21 to 30 bouw per week 

D. 31 to 40 hours per week 

E. more than 40 hours per week 

40. How many credits are you taking this semesterr 

A. U or fewer 

B. 12 or more 

41. Have you akoi a course with this instrucmr before* 

A. \fcS 

B. No 

C. I don t remember 

42. This questioimaire is 

A. Too long 

B. About the right length 

C. Tbo short 

43. This questionnaire was 

A. Easy to understand 

B. Hard to undentand 



35 How old are ^-ou? 

A. 18 or under 

B. 19 to 24 

C. 25 to 31 

D. 52 to 40 

E. 4l or over 



on the lines beiow. please make anv SUGGESTIONS vou have on the Q^'^STIONNAmE itself ^^^^ - areas 
vou «.oufd like to see tn the future or changes m wonimg that may make ,t eas.er to understand. 



Questionnaire continues on the next p2ge 
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Appendix B 
Details on Methodotogy 

Factor Analysis 

The purpose of the factor analysts was to find the common factor(s) underlying the 
Student Feedback Survey. The Statistical Analysis System (SAS) and maximum-likelihood 
(ML) method (see Lawley & Maxwell, 1971) were used to obtain the estimates and to test 
hypotheses about the number of common factors to be retained. Prior to running the factor 
analysis, the raw data were transformed to a correlation matrix to save computer time. In 
this step, data for respondents who left blank one or more of the 23 items were eliminated 
from further analysis. This reduced the data to 11,794 observations. 

The method for establishing the prior communality estimates involved setting the 
estimate for each variable to its squared muhiple correlation with all other variables. Both 
orthogonal and oblique rotations of the factors were employed. After factors were extracted 
using the Maximum Likelihood Method, they were prerotated using Varimax. This was 
followed by the Promax rotation. The resulting standardized regression coefficients from the 
rotated factor pattern were reported and discussed in the text. 

To decide on the number of factors to retain, several statistical tests were employed 
including Akaike's Information Criterion, Schwarz's Bayesian Criterion, and Tucker and 
Lewis's Reliability C:oefficient In general, the best factor solution has been reached when 
the first two measures are at their minimum and the last is at its maximum. In this analysis, 
this point was reached at eight factors. When a nine-factor solution was tried, communalities 
exceeded 1.0, indicating a lack of fit. and further analysis could not be done. 

To compute factor scores from the 8-factor solution, two approaches were used. One 
was to output the actual factor scores that the program computed from the factor analysis 
into ano'ther'data set. This approach applied weightings to every item for each factor. The 
second approach was to select the items for each factor that loaded above .30 (see Table 
4) and simply sum the responses to '.hose items to obtain the factor score. Correlations 
between the two approaches ranged from .95 to .98 depending on the factor, indicating the 
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results obtained by the two methods were very simUar. Because the second approach was 
simpler and more intuitively obvious, further anafyses were conducted in this manner. 

AHt^IY^I^"^ Variance 

The Statistical Analysis System (SAS) and the General Linear Models (GLM) 
Procedure were employed for this analysis. To begin, factor scores were standardized so 
that the mean was 0.0 and the standard deviation was 1.0 for each factor. Because a "1" or 
"strongly agree" was more positive than a "4" or strongly disagree, the result of this 
standardization process was that a negative number (e.g., -1.25) was more positive than a 
higher number (e.g., 1.3). Thus, the signs were switched to facilitate interpretation and 
positive numbers corresponded to positive signs. 

The analysis proceeded by first testing for the hypothesis of no overall effect among 
the eight factors using a multivariate analysis of variance (I%4AN0VA) and an alpha level 
of .05. If statistical significance was reached, then an analysis of variance (ANOVA) was 
performed for each factor using an alpha level of .01. If statistical significance was obtained, 
Tukey's Studentized Range (HSD) Test was employed to see which pairs of means were 
statistically significant at the .05 level. The difference between a pair of means was 
considered "educationally significant" if it was 0.5 or more. 
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